• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Computer-implemented method for generating a panoramic image intended for stereoscopic reproduction. According to the invention, the method comprises the following steps: a) taking at least one pair of images with a camera arrangement (1), wherein each pair of images consists of a first image and a second image and wherein a receiving position (7a) of the first image relative to a Receiving position (7b) of the second image is shifted and / or rotated; b) rotating the camera arrangement (1) with respect to a first rotation axis (3) by a rotation angle increment (D) from a rotation angle position (4a) to a next rotation angle position (4b); c) repeating steps a) and b) until a, preferably predeterminable, first rotation angle range (6) is swept over in order to produce a sequence of recordings of the at least one pair of images for the successive rotational angle positions (4a, 4b, 4c, ..., 4n) ,
    公开号:AT518256A4
    申请号:T50701/2016
    申请日:2016-08-02
    公开日:2017-09-15
    发明作者:
    申请人:Innaq Gmbh；
    IPC主号:
    专利说明:

    GENERATING A PANORAMIC IMPOSITION FOR STEREOSCOPIC REPRODUCTION AND SUCH A PLAYBACK
    FIELD OF THE INVENTION
    The present invention relates to a computer-implemented method for generating a panoramic image intended for stereoscopic reproduction.
    Moreover, the present invention relates to a computer-implemented method for the stereoscopic reproduction of a panoramic image recorded by means of a computer-implemented method according to the invention by means of a stereoscopic reproduction apparatus.
    STATE OF THE ART
    Panoramic photographs have been made for a long time in order to capture scenes such as To capture landscapes or real estate as an all-round view. A user or viewer of the panoramic image should be able to freely choose a viewing angle in the sequence, the viewing angle being directed to a specific part of the scene.
    Today, panoramic images are created by combining (also known as "stitching") individual images into an overall representation, typically taking the images in a horizontal rotation angle of 360 ° and an angle of elevation of 180 ° Projection methods (eg spherical or cubic) are shown in such a way that the observer obtains the image matching the desired viewing direction and the appropriate image detail The panorama images thus generated are used for monoscopic as well as for stereoscopic representations, which is associated with a number of disadvantages.
    The main disadvantage is that when the viewing angle changes, the motion parallax visible in reality can not be reproduced. Such changes in the viewing angle are particularly common in virtual reality applications where the user can turn and expect to see a corresponding rotation of the viewed panoramic image from the first person perspective. The lack of motion parallax causes an unnatural impression, which usually significantly reduces the immersion and the spatial impression of the user.
    In particular, in a stereoscopic - but also in a monoscopic - presentation, this has a negative effect.
    Another disadvantage of known production methods of panoramic images is the susceptibility to changes in the scene. A typical example of such a change would be a vehicle driving through the scene during shooting so that different parts of the vehicle (or no parts on some of the pictures) can be seen on different shots. This leads to the consequence that the recordings can not be combined or only with very limited quality.
    OBJECT OF THE INVENTION
    It is an object of the present invention to provide a method for generating panoramic images intended for stereoscopic reproduction and a stereoscopic reproduction method for these panoramic images which avoid the above-mentioned disadvantages. In particular, it should be possible to reproduce a motion parallax. Preferably, the method of generating the panoramic images shall be robust to dynamic changes, e.g. be opposed to moving vehicles, in scenes to be recorded.
    PRESENTATION OF THE INVENTION
    To achieve the object mentioned, a computer-implemented method for generating a panoramic image intended for stereoscopic reproduction is provided according to the invention, the method comprising the following steps: a) taking at least one pair of images with a camera arrangement, each pair consisting of a first image and a second image and wherein a photographing position of the first image is shifted and / or rotated relative to a photographing position of the second image; b) rotating the camera assembly with respect to a first axis of rotation by a Drehwinkelinkrement from a rotational angular position to a next rotational angular position; c) repeating steps a) and b) until a, preferably predeterminable, first rotation angle range is swept over in order to produce a sequence of recordings of the at least one pair of images for the successive rotational angle positions.
    It is provided in a preferred embodiment of the computer-implemented method according to the invention that the method comprises the following further step: d) generating at least one video file from the images of the sequence, preferably a first video file from the first images of the sequence and a second video file from the second pictures of the sequence. This is mainly the compression of the individual images and reduces the memory requirements accordingly. In principle, however, this can also be dispensed with and the generated sequence of image pairs can be used as such for the panoramic image. Of course, the two video files or the pair of video files can be packed into a single video file to better handle the panoramic image. Respectively. It is also possible to create a single video file directly from the pictures in the sequence.
    The panoramic image thus taken is suitable for stereoscopic reproduction, especially in virtual reality (VR) applications or augmented reality (AR) applications, e.g. by means of a so-called Head Mounted Display (HMD), which is also referred to as video glasses, helmet display, VR helmet or data helmet depending on the configuration.
    The images are taken in each case at certain rotational angle positions in the first rotation angle range.
    By taking a plurality of pairs of images per rotational angle position, a subsequence of pairs of images, in particular a pair of video sequences, can be recorded at each rotational angular position.
    Of course, it is also possible to record only one pair of images per angular position. Accordingly, it is provided in a preferred embodiment of the computer-implemented method according to the invention that exactly one pair of images is recorded for each rotational angular position, which u.a. a particularly fast recording of the panoramic image allows.
    The two images of each pair of images are taken from different recording positions, wherein the relative arrangement of the recording positions to each other is preferably the same at all rotational angle positions. This creates the prerequisite for the generation of motion parallax. The reproduction does not necessarily have to be stereoscopic, but can in principle also be monoscopic. The inventive method always provides a panoramic image with motion parallax.
    By choosing this relative arrangement to be similar to the relative arrangement of a user's eyes, the first image of each image pair can be basically assigned to one user's eye and the second image of each image pair to the other user's eye. This creates the prerequisite for stereoscopic vision.
    From the sequence of individual pairs of images following each other according to the successive rotational angle positions, each pair of images consisting of a first and a second image, two video files can result, namely a video file for the sequence of the first images and a video image. File for the sequence of second images. If several pairs of images are recorded per rotational angle position, a partial sequence of a plurality of first images and a partial sequence of several second images are present at each rotational angular position. Again, ultimately two video files can be obtained. One of these video files results from the sequence of the partial sequences of the first images, that is, again from a sequence of the first images. The other of these video files results from the sequence of the subsequences of the second images, that is, again from a sequence of the second images. In the simplest case, no further rotations are made about further axes of rotation, so that the generated panoramic image consists of the sequence of image pairs or of at least one video file, preferably of said two video files.
    The video files can basically be stored in video formats known per se and edited and reproduced with known tools or programs.
    Basically, this method of producing panoramic images in monoscopic playback is tolerant of changes in scenery since there is no need to stitch and each eye sees the same image.
    Preferably, the first rotation axis is parallel to the vertical, so that a horizontal first rotation angle range results.
    The first rotation angle range preferably extends from a rotation angle start position to a rotation angle end position.
    The first rotation angle range may be e.g. 360 °. Typical values for the rotational angle increment are in the range from 0.01 ° to 20 °, preferably from 0.1 ° to 5 °, particularly preferably from 0.5 ° to 3 °.
    Preferably, the Drehwinkelinkrement is chosen to be constant. In this case, the rotational angle positions can be determined as the sum of the rotational angle start position and an integral multiple of the rotational angle increment.
    In order to be able to ensure the two pickup positions with as little constructive effort as possible, in particular as inexpensively as possible, at each rotational angle position, it is provided in a preferred embodiment of the computer-implemented method according to the invention that the camera arrangement comprises a camera which is disposed between the camera
    Recording position of the first image and the recording position of the second image is movable back and forth. Of course, this is to be understood that the two shooting positions can actually be taken by the camera. It follows that the panoramic image according to the invention can also be generated with only a single camera.
    The movement of the camera between the shooting positions can be done in different ways. In a preferred embodiment of the computer-implemented method according to the invention, it is provided that the movement of the camera between the recording position of the first image and the recording position of the second image is effected by a linear displacement. For this purpose, the camera may be guided on a linear rail, for example, and e.g. be moved by means of an electric motor or by means of an electric, pneumatic or hydraulic cylinder. The linear rail in turn can be rotated about the first axis of rotation to transfer the camera assembly from one rotational position to the next.
    In order to realize the movement of the camera between the recording positions technically particularly simple, in particular particularly cost-effective, it is provided in a preferred embodiment of the computer-implemented method according to the invention that the movement of the camera between the recording position of the first image and the recording position of the second image a rotation, preferably about the first axis of rotation takes place.
    In particular, when the rotation about the first axis of rotation takes place, the same technical means can be used to on the one hand to transfer the camera assembly from one rotational position to the next and on the other hand to move the camera between the two pickup positions. In order to take account of the spaced-apart arrangement of the human eyes, two image parts or "image halves" are used at each rotational angle position, which may preferably overlap Further, to generate the pair of images at a certain rotational angle position, images are taken of rotational angular positions that come before and after the determined rotational angular position, for example, to generate the image pair at the determined rotational angular position on the one hand, the image, which is recorded at two rotational angle positions previously, and on the other hand, the image, which is recorded at two rotational angular positions subsequently used.
    Preferably, to generate the pair of images at the specific rotational angle position, on the one hand the image which is recorded at the immediately preceding rotational angle positions and on the other hand the image which is recorded at the immediately following rotational angular positions are used. By way of example, reference will be made to this special case in the following, but the explanation applies, of course, analogously to the aforementioned general case (rotational angle positions which come before and after the determined rotational angular position): the one image part of the image which is at the rotational angular position immediately before the determined rotational angular position is recorded, the first image of the image pair is at the specified rotation angle position.
    The recording position of the first image is thus set by the camera arrangement at the rotational angle position immediately before the determined rotational angular position. The other image part of the image taken at the rotational angular position immediately after the determined rotational angular position is the second image of the image pair at the determined rotational angular position. The recording position of the second image is thus determined by the camera arrangement at the rotational angle position immediately after the determined rotational angular position.
    In other words, the camera takes on one
    Rotation angle position with its "left" image section on the image that would have seen the right eye of a user at a previous position, and in the "right" section is the image for the left eye, which would have seen it at a later position.
    According to the above, step b) can also be carried out at least partially simultaneously with step a), which in turn allows a time-saving recording of the panoramic image.
    In a preferred embodiment of the computer-implemented method according to the invention, the images are taken with a wide-angle objective, preferably with a fisheye objective, in order to cover the largest possible area with each image.
    In order to provide an even faster and structurally more stable possibility for the recordings, it is provided in a preferred embodiment of the computer-implemented method according to the invention that the camera arrangement comprises a first camera and a second camera displaced and / or rotated relative thereto, the first image is taken with the first camera and the second image with the second camera. Preferably, the first camera takes all first images and the second camera takes all second images.
    Preferably, the triggering of the two cameras takes place simultaneously, i. the first picture and the second picture are recorded simultaneously.
    From the sequence of first images taken with the first camera, the first video file can be generated and from the sequence of second images taken with the second camera, the second video file.
    The two cameras can for this purpose e.g. be mounted on a holder, wherein the position of the two cameras to each other on the holder is preferably adjustable. The holder may then be rotated about the first axis of rotation to transfer the camera assembly from one rotational position to the next. During the implementation of the method according to the invention, the relative arrangement of the two cameras - and thus in particular their distance from one another - preferably remains fixed.
    The version with two cameras is also tolerant to changes in scenery even in stereoscopic playback. For example, if a car moves through the scene during recording, it is usually captured by both cameras at the same time. In stereoscopic reproduction, therefore, the car is also displayed to each eye, i. it does not occur that e.g. the left eye sees a part of the car and the right eye sees no part of the car.
    If the recordings are made slowly at the successive rotational positions with the corresponding rotation, it can only happen that the car is visible at a rotational angle position and not at an adjacent rotational angular position, but this does not significantly disturb the immersion or the spatial impression. If the recordings are made quickly at the successive rotational positions with the corresponding rotation, the movement of the car may be traced to some extent. That the user sees the car at different positions in the scenery when viewing the panoramic image at the different rotational angle positions and perceives this as movement of the car, which can additionally enhance the immersion or the spatial impression.
    In other words, the rapid recording with two cameras even enables the recording of changes in the scene and stereoscopic reproduction.
    As already stated, the arrangement of the two recording positions relative to one another can be selected such that there is a certain similarity to the relative arrangement of the eyes of a user for the particular scene. It should be noted that a line that connects the two eyes of the user with each other, when the user rotates about an axis of rotation generally does not intersect the rotation axis, but will be spaced therefrom. To take this circumstance into account when taking the panoramic image, it is provided in a preferred embodiment of the computer-implemented method according to the invention that the camera arrangement has a first distance from the first axis of rotation. In other words, the camera assembly is rotated eccentrically about the first axis of rotation.
    Preferably, the first distance on the one hand and the relative arrangement of the two receiving positions to each other on the other hand are matched. The panoramic picture thus created allows a particularly realistic-looking playback. In the case of VR applications, it is thus possible to achieve a particularly high quality immersion of the user.
    In order to cover the largest possible area with the panoramic image, which corresponds to a rotation about a second axis of rotation, which second axis of rotation with the first axis of rotation includes an angle not equal to zero, preferably an angle of 90 °, a fisheye lens can be used in the recording. In this way, with each shot practically a range of 180 ° can be recorded around the second axis of rotation. In particular, it is possible in this way to cover a vertical area, without an actual pivoting of the camera assembly about the second axis of rotation, which would then be arranged horizontally, would be necessary.
    Of course, but also when using a
    Fisheye lens rotation about the second axis to cover an even larger area around the second axis. Preferably, however, a rotation about the second axis is used when using lenses that are not fisheye lenses.
    In a preferred embodiment of the computer-implemented method according to the invention, it is provided that steps a) to c) are repeated after the camera arrangement has been rotated about a second axis of rotation, wherein the second axis of rotation with the first axis of rotation is at a non-zero angle, preferably an angle of 90 °. It is preferably provided that the steps a) to d) are repeated after the camera arrangement has been rotated about the second axis of rotation. As a result of the rotation or possibly the rotations about the second rotation axis, a sequence of second rotation angle positions (relative to the second rotation axis), preferably a sequence of elevation angles, is set or generated. That in the case of a plurality of second rotational-angle positions about the second rotational axis, a sequence of image pairs or respectively preferably the first and second video-file are respectively recorded. In other words, the captured panoramic image consists of multiple sequences of video files, the number of these sequences or pairs being equal to the number of second angular position taken up, and each of the pairs being one of the first video files and the associated one second video file exists.
    In particular, a vertical component of the scene to be captured can also be taken into account in this way by the panoramic image not only covering a horizontal rotation angle range, but also an elevation angle range. The elevation angle range is therefore a special case of a second rotation angle range in which the second rotation angle positions are located. This extends from a second angle of rotation starting position to a second angle of rotation end position. At least one elevation angle (sometimes just referred to as height or elevation) or a second angular position is always present anyway.
    Of course, a plurality of repetitions may take place so that not only two (i.e., not only the second rotational angle start position and the second rotational angle end position) but also more than two second rotational angular positions which are in the second rotational angular range are adopted.
    The second rotation angle range or its size may be e.g. in the range of 45 ° to 180 °, preferably from 45 ° to 90 °. But there are also larger areas conceivable.
    Successive second rotational angle positions may differ by one, preferably constant, second rotational angle element. Typical values for the second rotation angle increment are in the range from 0.01 ° to 20 °, preferably from 0.1 ° to 5 °, particularly preferably from 0.5 ° to 3 °.
    As mentioned, the panoramic images produced according to the invention are ideally suited for stereoscopic reproduction. Accordingly, the present invention provides a computer-implemented method of stereoscopically reproducing a panoramic image captured by a computer-implemented method according to the present invention by means of a stereoscopic display device, the method comprising the steps of: aa) determining a viewing angle at which the panoramic image is to be displayed; bb) calculating a display rotation angle position, wherein the display rotation angle position is the rotation angle position corresponding to the view angle with respect to the first rotation axis; cc) determining the rotational angular position in the sequence closest to the rotational latent position; dd) using the at least one pair of images of this rotational angle position for the creation of at least one display image pair; ee) reproducing the at least one first image of the at least one display image pair by means of a first display of the stereoscopic display device and the at least one second image of the at least one display image pair by means of a second display of the stereoscopic display device.
    A user sees with one eye the first display of the stereoscopic display device and with the other eye the second display. When changing the angle of view, the motion parallax is fully reproduced, resulting in a particularly realistic viewing experience for the user.
    The use of the at least one image pair, which was taken for the rotational angle position in the sequence that comes closest to the representation rotational angle position, for the creation of the at least one representation image pair can be done differently. In a particularly simple embodiment of the computer-implemented method according to the invention, it is provided that in step dd) the at least one pair of images of this rotational angle position is adopted as the at least one representation image pair.
    In particular, if the rotational angle positions are not very close around the first axis of rotation, that is, if the Drehwinkelinkrement has been relatively large in the images, as continuous as possible course of the panoramic image or a possible correct representation of the panoramic image at the desired
    Display rotational position can be interpolated or computationally achieved. For this purpose, it is provided in a preferred embodiment of the computer-implemented method according to the invention that the following steps are performed in step dd): determining a deviation of this rotational angle position from the representation rotational angular position; if the deviation is greater than a predefinable threshold value, calculating the at least one representation image pair from the at least one image pair of that rotation angle position which occurs in the sequence immediately before
    Representation rotational position is, and the at least one pair of images that rotational angle position, which is in the sequence immediately after the presentation rotational position; otherwise: Applying the at least one image pair of this rotational angle position as the at least one representation image pair. The latter corresponds to the above-mentioned particularly easy to implement embodiment. In the event that the panorama image is in the form of at least one video file, preferably in the form of at least one pair of video files, it should be noted that the sequence of image pairs is of course directly from the at least one video file, preferably from which at least one pair of video files is extracted or extracted, and thus in any case is available to perform the above steps.
    The calculation of the at least one representation image pair can be carried out in a manner known per se, for example by blending the first images of the at least one image pair at the rotation angle position which lies immediately before the presentation rotation angle position in the sequence, and the at least one image pair at that rotation angle position which is shown in FIG Sequence immediately after
    Display rotational position is. However, known methods are also applicable in which the images are projected onto a projection body (usually a sphere or a cube), the projection body being rotated and / or translated. Respectively. Of course, combinations of such methods are also applicable.
    The determination of the angle of view can be done in different ways. In order to enable a determination that is technically particularly easy to implement, in a preferred embodiment of the computer-implemented method according to the invention it is provided that the determination of the viewing angle is based on a choice made by means of an input device. That the user can e.g. Using a keyboard, a mouse, a touch screen or the like, enter or select the desired angle.
    In particular, in order to enable the user with a high-quality immersion in VR applications, it is provided in a preferred embodiment of the computer-implemented method according to the invention that the determination of the viewing angle takes place on the basis of sensor signals. That A conscious input of the user is not absolutely necessary. Instead, movements of the user are detected by means of sensors and the corresponding sensor signals are evaluated. Such sensor systems are known per se from today's VR applications. For example, At least one so-called eye tracker can be used to
    To follow movement of an eye or the eyes of the user and to determine thereon based on the current angle of view. Or it can e.g. a so-called head tracker can be used to register movements of the user's head and based on this determine the current viewing angle. Furthermore, e.g. the orientation of other parts of the body or of the entire body of the user can be determined continuously by means of the sensors in order to determine the current viewing angle based thereon. In the case where the panoramic image, which has been produced with the computer-implemented method according to the invention, contains images at more than one elevation angle or at more than one second rotational angle position, it is provided in a preferred embodiment of the computer-implemented method according to the invention, that also has a second Rotation angle position is determined, which corresponds to the angle of view relative to the second axis of rotation, that sequence is determined which can be assigned to a second rotational angular position with respect to the second axis of rotation, which second rotational angle position of the second representation rotational position comes closest, and that the image pairs of this sequence for the creation of the at least one display image pair can be used.
    In the simplest case, only the image pairs of this sequence are used to create the at least one display image pair.
    However, it is also possible to use image pairs from "adjacent" sequences (related to the second rotational angle positions) for producing the at least one display image pair, in particular if the second rotational angular positions around the second rotational axis are not very dense, ie if the second rotational angle increment has been relatively large In the case of a preferred embodiment of the computer-implemented method according to the invention, when using the image pairs of this sequence, it is provided that the sequence of the panorama image should be as continuous as possible or the representation of the panorama image should be as correct as possible the creation of the at least one display image pair following steps are provided: - Determining a deviation of this second rotation angle position of the second representation rotation angle position; - If the deviation ng is greater than a predefinable second threshold value: calculating the at least one display image pair from the at least one image pair of the sequence at that second rotation angle position which is immediately before the second representation rotation angle position, and the at least one image pair of the sequence at that second rotation angle position immediately after second presentation rotational position is; otherwise: Apply the image pairs of the sequence at this second rotation angle position to create the at least one representation image pair. The latter corresponds to the above-mentioned particularly easy to implement embodiment.
    The calculation of the at least one representation image pair can be carried out in a manner known per se, for example by blending the image pairs of the sequences which are at the second rotation angle positions, the second
    Adjacent display rotation position. However, known methods are also applicable in which the images are projected onto a projection body (usually a sphere or a cube), the projection body being rotated and / or translated. Respectively. Of course, combinations of such methods are also applicable.
    BRIEF DESCRIPTION OF THE FIGURES
    The invention will now be explained in more detail with reference to exemplary embodiments. The drawings are exemplary and are intended to illustrate the inventive idea, but in no way restrict it or even reproduce it.
    Showing:
    1 shows a schematic illustration of a camera arrangement when carrying out a method according to the invention, the camera arrangement having a camera
    2 shows a schematic illustration of a camera arrangement when carrying out a method according to the invention, the camera arrangement having a first camera and a second camera
    WAYS FOR CARRYING OUT THE INVENTION
    1 illustrates a camera arrangement 1 as used in the implementation of a computer-implemented method according to the invention for generating a panoramic image intended for stereoscopic reproduction. In principle, the computer-implemented method according to the invention comprises the following steps: a) taking at least one pair of images with the camera arrangement 1, each pair of images consisting of a first image and a second image, and a receiving position 7a of the first image relative to a receiving position 7b of the second image shifted and / or twisted; b) rotating the camera assembly 1 with respect to a first rotation axis 3 by a rotation angle increment Δ from a rotation angle position 4a to a next rotation angle position 4b; c) repeating steps a) and b) until a, preferably predefinable, first rotation angle range 6 is swept over in order to produce a sequence of recordings of the at least one pair of images for the successive rotational angle positions 4a, 4b, 4c, ..., 4n.
    Preferably, the following additional step is taken for reasons of space: d) generating at least one video file from the pictures of the sequence, preferably a first video file from the first pictures of the sequence and a second video file from the second pictures of the sequence.
    As a result, in particular the reproduction of a motion parallax is made possible.
    To reproduce the thus captured panoramic image by means of a stereoscopic display device, such as a video recorder. by means of a head-mounted display (not shown), a computer-implemented method is provided according to the invention, comprising the following steps: aa) determining a viewing angle at which the panoramic image is to be reproduced; bb) calculating a display rotation angle position, wherein the display rotation angle position is the rotation angle position corresponding to the view angle with respect to the first rotation axis 3; cc) determining the rotational angular position in the sequence closest to the rotational latent position; dd) using the at least one pair of images of this rotational angle position for the creation of at least one display image pair; ee) reproducing the at least one first image of the at least one display image pair by means of a first display of the stereoscopic display device and the at least one second image of the at least one display image pair by means of a second display of the stereoscopic display device. A user sees with one eye the first display of the stereoscopic display device and with the other eye the second display. When changing the viewing angle, e.g. when the user turns and the rotation is sensed by sensors or when the user selects a different viewpoint by means of an input device, the motion parallax is fully reproduced, giving a particularly realistic viewing experience to the user.
    In the exemplary embodiment of FIG. 1, the camera arrangement 1 comprises a camera 9.
    The camera assembly 1 is rotatable about the first axis of rotation 3, for example by means of an electric motor (not shown). The first rotation axis 3 may-but need not of course-be parallel to the vertical, so that in this case the first rotation angle range 6 is a horizontal first rotation angle range 6.
    The first rotation angle range 6 is approximately 305 ° in FIG. 1, but can in principle also be up to and including 360 °.
    The rotation is eccentric to the first axis of rotation 3, i. the camera arrangement 1 has a first distance 5 greater than zero from the first axis of rotation 3. For example, the first distance 5 can be in the range from 1 cm to 100 cm, preferably from 5 cm to 15 cm. Typically, the distance becomes similar to the radius of a human head, e.g. approx. 8 cm, chosen to capture the user's perspective as realistically as possible. However, it is possible to take pictures in a way that looks like a dwarf or a giant, for which more extreme distances are chosen, which may be in the ranges given above.
    In Fig. 1, the camera 9 is drawn with solid lines at the rotational angle position 4a, wherein the rotational angle position 4a corresponds to a rotational angle start position. The rotation angle position 4n thus corresponds to a rotation angle end position, wherein the first rotation angle range from the rotation angle start position to the
    Angle of rotation end position extends. At the rotational angle positions 4b, 4c, ..., 4n, the camera 9 is indicated only by dashed lines.
    The Drehwinkelinkrement Δ is chosen constant in the embodiment of FIG. 1 and is about 12.5 °.
    At each rotation angle position 4a, ..., 4n, a picture is taken. In order to take account of the spaced-apart arrangement of the human eyes, the recorded image at each rotation angle position 4a, ..., 4n divided into two image parts that can overlap each other.
    Furthermore, to generate the image pair at a certain rotational angle position, images are taken of rotational angle positions which come before and after the determined rotational angular position. For example, to generate the pair of images at the specific rotational angle position, on the one hand, the image which is previously recorded at two rotational angle positions and, on the other hand, the image which is recorded at two rotational angle positions are used. Preferably, to generate the pair of images at the specific rotational angle position, on the one hand the image which is recorded at the immediately preceding rotational angle positions and on the other hand the image which is recorded at the immediately following rotational angular positions are used.
    With reference to FIG. 1, this can be illustrated as follows, wherein the two thick lines in the camera 9 symbolize the two image parts of the respectively recorded image. For illustrative purposes, the two thick lines are spaced apart, but it should be emphasized again that the
    Image parts may actually overlap and generally overlap. The rotational angle position 4b is the specific rotational angular position. The one (in the illustrated example right) image part of the image, which is taken at the rotational angle position 4a immediately before the rotational angle position 4b, is the first image of the image pair to the rotational angle position 4b. The recording position 7a of the first image of the pair of images for the rotational angle position 4b is therefore set by the camera arrangement 1 at the rotational angle position 4a immediately before the rotational angular position 4b. The other (left-hand in the illustrated example) image part of the image, which is taken at the rotation angle position 4c immediately after the rotation angle position 4b, is the second image of the image pair to the rotation angle position 4b. The recording position 7b of the second image of the pair of images for the rotational angle position 4b is thus set by the camera arrangement at the rotational angle position 4c immediately after the rotational angular position 4b. For the generation of the other pairs of images to the other rotational angle positions is proceeded completely analog.
    The camera 9 is thus moved by rotation of the camera assembly 1 about the first axis of rotation 3 at the same time between the receiving positions 7a, 7b back and forth. This therefore represents a technically particularly simple solution to be realized. According to the above, a rotation angle α results around the first rotation axis 3, which is 1.5 times the rotation angle increment Δ, between the reception positions 7a, 7b.
    In the camera assembly 1, the camera 9 is further rotatably mounted about a second axis of rotation 8 and can be rotated about this in a conventional manner, for example with an electric motor (not shown). The second axis of rotation 8 is in the illustrated embodiment normal to the first axis of rotation 3. The second axis of rotation 8 may - but need not of course - be parallel to the horizontal, so surpassed by rotation of the camera assembly 1 about the second axis of rotation 8, an elevation angle range or a second rotation angle range which is typically in the range of 45 ° to 180 °.
    It should be noted that the elevation angle range could also be covered by using a fisheye lens in order, if necessary, to be able to dispense with a rotation of the camera arrangement 1 about the second axis of rotation 8.
    Specifically, in the illustrated embodiment, the recording of the panoramic image can be carried out so that at a plurality of second rotational angle positions about the second axis of rotation 8 in each case the sequence of image pairs or the first and second video file are recorded, as described above. In this case, the first rotation angle range 6 is preferably completely swept over at every second rotational angular position. That the pairs of images or images are recorded at the rotational angle positions 4a,..., 4n at a certain second rotational angle position, then the rotation about the second rotational axis 8 takes place by a second rotational angle increment, whereupon the image pairs or images at the rotational angle positions 4a, ..., 4n be recorded and so on. The panoramic image therefore comprises a sequence of image pairs or a first and a second video file at every second rotational angle position or for every second rotational angular position. In the event that the panoramic image contains images at more than one elevation angle or at more than one second angular position, a second representation rotational position can be determined during playback, which corresponds to the viewing angle with respect to the second axis of rotation 8. Then, that sequence is determined which can be assigned to that second rotational angular position which comes closest to the second rotational representation angular position. The
    Image pairs of this sequence are finally used to create the at least one display image pair.
    FIG. 2 shows a further embodiment variant of the camera arrangement 1, as used in the implementation of a computer-implemented method according to the invention for generating a panoramic image intended for stereoscopic reproduction. In contrast to FIG. 1, the camera arrangement 1 in FIG. 2 has a first camera 2a and a second camera 2b, which permits a particularly rapid acquisition of the panoramic image. The first camera 2a and the second camera 2b are mounted on a linear mount 10 so that they have a preferably adjustable, fixed offset arrangement to each other. Typically, the two cameras 2a, 2b have a distance 11 in the range from 1 cm to 100 cm, preferably from 3 cm to 16 cm to each other. Typically, the distance will be similar to the distance of the human eyes, e.g. approx. 6.4 cm, chosen to capture the user's perspective as realistically as possible. However, it is possible to take pictures in a way that looks like a dwarf or a giant, for which more extreme distances are chosen, which may be in the ranges given above. It should be noted that theoretically it would also be conceivable that the cameras 2a, 2b can also be rotated relative to each other.
    The first camera 2 a serves to receive the first images and is located at the recording position 7 a at each rotational angle position 4 a,..., 4 n. The second camera 2b is for receiving the second images and is located at each rotational angle position 4a, ..., 4n at the pickup position 7b. Due to the relatively shifted arrangement of the cameras 2a, 2b, the receiving positions 7a, 7b are shifted relative to each other accordingly. From the sequence of first images taken with the first camera 2a and the sequence of second images taken with the second camera 2b, the sequence of image pairs can be generated. Respectively. For example, from the sequence of first images taken with the first camera 2a, the first video file can finally be generated, and from the sequence of second images taken with the second camera 2b, the second video file.
    During rotation about the first axis of rotation 3, the two cameras 2 a, 2 b are rotated together with the linear holder 10. The linear holder 10 is arranged at the first distance 5 from the first axis of rotation 3. The first distance 5 is also in the embodiment of FIG. 2 in the range of 1 cm to 100 cm, preferably from 5 cm to 15 cm.
    The first rotation angle range 6 in FIG. 2 is 360 °, wherein the camera arrangement 1 is shown only with the rotation angle position 4b with solid lines and is indicated at the rotation angle positions 4a, 4c with dotted lines. The rotational angle increment Δ is selected to be very large in FIG. 2 and is approximately 39 °.
    In the embodiment of FIG. 2 extends through the linear support 10, the second axis of rotation 8, which is also normal in this case on the first axis of rotation 3. In this case, the second axis of rotation 8 - as in the embodiment of FIG. 1 - at each rotational angle position 4a, ..., 4n a tangent to an imaginary circle 12 whose radius is the first distance 5 and whose center is located on the first axis of rotation 3 , That The camera arrangement 1 of FIG. 2 also permits the coverage of the first rotation angle range 6 about the first rotation axis 3 and of the second rotation angle range about the second rotation axis 8. As already described above, during the recording, the procedure is typically such that at every second rotation angle position the first Rotation angle range 6 is completely swept over. That at a certain second rotational angle position, the image pairs are recorded at the rotational angle positions 4a, 4n, then the rotation about the second rotational axis 8 is effected by a second rotational angle increment, whereupon again the image pairs are recorded at the rotational angle positions 4a,..., 4n and so on , The panoramic image therefore comprises a sequence of image pairs or a first and a second video file at every second rotational angle position or for every second rotational angular position, wherein each first video file has been recorded with the first camera 2a and every second video file with the second camera 2b.
    REFERENCE LIST I Camera arrangement 2a First camera 2b Second camera 3 First rotation axis 4a, 4b, 4c,..., 4n Rotation angle position 5 First distance 6 First rotation angle range 7a, 7b Acquisition position of first / second image 8 Second rotation axis 9 Camera 10 Linear support II Distance between The first and the second camera 12 Thought circle Δ rotation angle increment CC Rotation angle between the shooting positions of the first and second images
    权利要求:
    Claims (16)
    [1]
    A computer-implemented method for generating a panoramic image intended for stereoscopic reproduction, the method comprising the following steps: a) taking at least one pair of images with a camera arrangement (1), wherein each pair of images consists of a first image and a second image and wherein a recording position (7a) of the first image is shifted and / or rotated relative to a pickup position (7b) of the second image; b) rotating the camera arrangement (1) with respect to a first rotation axis (3) by a rotation angle increment (Δ) from a rotation angle position (4a) to a next rotation angle position (4b); c) repeating steps a) and b) until a, preferably predefinable, first rotation angle range (6) is swept over in order to obtain a sequence of recordings of the at least one pair of images for the successive rotational angle positions (4a, 4b, 4c, ..., 4n) to create.
    [2]
    2. A computer-implemented method according to claim 1, characterized in that the method comprises the further step of: d) generating at least one video file from the images of the sequence, preferably a first video file from the first images of the sequence and a second video File from the second images of the sequence.
    [3]
    A computer-implemented method according to any one of claims 1 to 2, characterized in that the camera arrangement (1) comprises a camera (9) which reciprocates between the pickup position (7a) of the first picture and the pickup position (7b) of the second picture is movable.
    [4]
    4. A computer-implemented method according to claim 3, characterized in that the movement of the camera (9) between the recording position (7a) of the first image and the recording position (7b) of the second image is effected by a linear displacement.
    [5]
    5. A computer-implemented method according to claim 3, characterized in that the movement of the camera (9) between the receiving position (7a) of the first image and the receiving position (7b) of the second image by rotation, preferably about the first axis of rotation (3), he follows.
    [6]
    6. Computer-implemented method according to one of claims 1 to 2, characterized in that the camera arrangement (1) comprises a first camera (2a) and a relative to this shifted and / or rotated second camera (2b), wherein the first image with the first camera (2a) and the second image with the second camera (2b).
    [7]
    7. Computer-implemented method according to one of claims 1 to 6, characterized in that for each rotational angular position (4a, 4b, 4c, ..., 4n) exactly one pair of images is recorded.
    [8]
    8. Computer-implemented method according to one of claims 1 to 7, characterized in that the camera arrangement (1) from the first axis of rotation (3) has a first distance (5).
    [9]
    9. A computer-implemented method according to any one of claims 1 to 8, characterized in that the steps a) to c) are repeated after the camera assembly (1) has been rotated about a second axis of rotation (8), wherein the second axis of rotation (8) with the first axis of rotation (3) an angle not equal to zero, preferably an angle of 90 °, includes.
    [10]
    A computer-implemented method of stereoscopically reproducing a panoramic image captured by a computer-implemented method according to any of claims 1 to 9 by means of a stereoscopic display device, the method comprising the steps of: aa) determining a viewing angle at which the panoramic image is to be displayed; bb) calculating a display rotation angle position, wherein the display rotation angle position is the rotation angle position corresponding to the viewing angle with respect to the first rotation axis (3); cc) determining the rotational angular position in the sequence closest to the rotational latent position; dd) using the at least one pair of images of this rotational angle position for the creation of at least one display image pair; ee) reproducing the at least one first image of the at least one display image pair by means of a first display of the stereoscopic display device and the at least one second image of the at least one display image pair by means of a second display of the stereoscopic display device.
    [11]
    11. Computer-implemented method according to claim 10, characterized in that at step dd) the at least one pair of images of this rotational angle position is adopted as the at least one representation image pair.
    [12]
    12. Computer-implemented method according to claim 10, characterized in that the following steps are performed at step dd): determining a deviation of this rotational angular position from the rotational representation position; if the deviation is greater than a predefinable threshold value, calculating the at least one representation image pair from the at least one image pair of the rotation angle position which lies immediately before the presentation rotation angle position in the sequence, and the at least one image pair that rotation angle position which follows in the sequence immediately after the representation rotation angle position lies; otherwise: Applying the at least one image pair of this rotational angle position as the at least one representation image pair.
    [13]
    13. Computer-implemented method according to one of claims 10 to 12, characterized in that the determination of the angle of view is made on the basis of a choice made by means of an input device.
    [14]
    14. Computer-implemented method according to one of claims 10 to 13, characterized in that the determination of the angle of view takes place on the basis of sensor signals.
    [15]
    15. Computer-implemented method according to claim 10, characterized in that, in addition, a second representation rotational angle position is determined, which corresponds to the viewing angle with respect to the second rotational axis (8), that sequence is determined which corresponds to a second rotational angular position relative to the second rotational angle Rotary axis (8) can be assigned, which second rotation angle position of the second representation rotation angle position comes closest, and that the image pairs of this sequence are used for the preparation of the at least one display image pair.
    [16]
    16. A computer-implemented method according to claim 15, characterized in that when using the image pairs of this sequence for the preparation of the at least one display image pair following steps are provided: - Determining a deviation of this second rotational angular position of the second Darstellungsdrehwinkelposition; if the deviation is greater than a predefinable second threshold value, calculating the at least one representation image pair from the at least one image pair of the sequence at that second rotation angle position which lies immediately before the second representation rotation angle position and the at least one image pair of the sequence at that second rotation angle position is immediately after the second display rotational position; otherwise: Apply the image pairs of the sequence at this second rotation angle position to create the at least one representation image pair.
    类似技术:
    公开号 | 公开日 | 专利标题
    EP2800350A2|2014-11-05|Assembly and method for recording and reproducing images of a scene and/or an object
    DE112016002377T5|2018-02-08|STREAMS OF SPHERICAL VIDEO
    DE602004008794T2|2008-06-12|IMAGE PLAYBACK WITH INTERACTIVE MOTION PARALLAX
    AT394459B|1992-04-10|METHOD FOR OBTAINING IMAGES FOR USE IN REPRESENTING A THREE-DIMENSIONAL SHEET IMAGE RECORDING CARRIER ON WHICH THE FIRST AND SECOND GROUPS OF SERIES OF SUCH IMAGES ARE STORED
    DE202017104934U1|2017-11-20|Multi-level camera carrier system for stereoscopic image acquisition
    DE102011008886A1|2012-07-19|Method and apparatus for stereobase extension of stereoscopic images and image sequences
    DE4123895A1|1993-01-21|Auto-stereoscopic image, film or television display system - uses controlled optical shutter in front of display screen for switching between two perspective images
    DE102015210453B3|2016-10-13|METHOD AND APPARATUS FOR GENERATING DATA FOR A TWO OR THREE-DIMENSIONAL PRESENTATION OF AT LEAST ONE PART OF AN OBJECT AND FOR PRODUCING THE TWO OR THREE-DIMENSIONAL PRESENTATION AT LEAST THE PART OF THE OBJECT
    AT518256B1|2017-09-15|GENERATING A PANORAMIC IMPOSITION FOR STEREOSCOPIC REPRODUCTION AND SUCH A PLAYBACK
    DE102016203709B4|2018-04-12|Image processing method, image processing means and image processing apparatus for generating images of a part of a three-dimensional space
    DE102015217226A1|2017-03-09|DEVICE AND METHOD FOR GENERATING A MODEL FROM AN OBJECT WITH OVERLOAD IMAGE DATA IN A VIRTUAL ENVIRONMENT
    DE102010041870A1|2012-04-05|Method for horizon-fit stereoscopic image processing of stereoscopic images or image sequences in fields of e.g. stereoscopic photography, involves creating two images based on picture and depth information with respect to viewing positions
    DE102018122435A1|2020-03-19|Virtual three-dimensional objects in a live video
    EP3139603A1|2017-03-08|System and method for producing and dispensing stereoscopic video films
    WO2012010117A1|2012-01-26|Method and device for recording three-dimensional image material for different display variables while utilizing the particular full depth budget
    EP3086554B1|2019-04-24|System and method for producing and dispensing stereoscopic video films
    DE102018202707A1|2019-08-22|Generation of panoramic pictures
    DE102017131083A1|2019-06-27|System and method for recording and visualizing a 3D scene in real time
    WO2020201571A1|2020-10-08|Method for an immersive display of stereoscopic images and image sequences
    DE10308810B4|2005-04-21|Method for the compressed transmission of image data for a 3-dimensional representation of scenes and objects
    DE102016102868A1|2017-08-24|System for displaying objects in a virtual three-dimensional image space
    WO2002075656A1|2002-09-26|Method and system for recording and representing three-dimensional objects
    EP1941745A1|2008-07-09|Method and device for the stereoscopic recording of objects for a three-dimensional visualization
    WO2013050514A1|2013-04-11|Film recording method that enables the production of sceneries having a three-dimensional effect
    DE102008025913A1|2009-12-03|Scene and/or object image recording and reproducing method for multi-focal camera system, involves providing combination of two different images for spatial representation, and representing combined three-dimensional image on display
    同族专利:
    公开号 | 公开日
    WO2018024503A1|2018-02-08|
    AT518256B1|2017-09-15|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20100097444A1|2008-10-16|2010-04-22|Peter Lablans|Camera System for Creating an Image From a Plurality of Images|
    US20160088280A1|2014-09-22|2016-03-24|Samsung Electronics Company, Ltd.|Camera system for three-dimensional video|DE102018202707A1|2018-02-22|2019-08-22|Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.|Generation of panoramic pictures|IL139995A|2000-11-29|2007-07-24|Rvc Llc|System and method for spherical stereoscopic photographing|
    法律状态:
    优先权:
    申请号 | 申请日 | 专利标题
    ATA50701/2016A|AT518256B1|2016-08-02|2016-08-02|GENERATING A PANORAMIC IMPOSITION FOR STEREOSCOPIC REPRODUCTION AND SUCH A PLAYBACK|ATA50701/2016A| AT518256B1|2016-08-02|2016-08-02|GENERATING A PANORAMIC IMPOSITION FOR STEREOSCOPIC REPRODUCTION AND SUCH A PLAYBACK|
    PCT/EP2017/068443| WO2018024503A1|2016-08-02|2017-07-21|Production of a panoramic image provided for a stereoscopic reproduction and such a reproduction|
    [返回顶部]