• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Wireless communication channel operation method and portable terminal system A voice-enabled function-enabled terminal and a voice-conversation control method are provided to provide distinct content based on the user's current emotional state, age and gender. the handset that supports a voice chat function includes a display unit, an audio processing unit, which selects content matching a user's first criteria in response to a user input, determines a content delivery scheme based on a second user-associated criterion, and provides the selected content through the display unit and audio processing unit according to the content delivery scheme.
    公开号:BR102014003021A2
    申请号:R102014003021-2
    申请日:2014-02-07
    公开日:2018-04-10
    发明作者:Ahn Jihyun；Kim Sora；Kim Jinyong；Kim Hyunkyoun；Kim Heewoon；Ahn Yumi
    申请人:Samsung Electronics Co, Ltd；
    IPC主号:
    专利说明:

    (54) Title: WIRELESS COMMUNICATION CHANNEL OPERATION METHOD AND PORTABLE TERMINAL SYSTEM (51) Int. Cl .: G06F 3/16; G06F 17/00; G06F 3/01; G06F 3/048; G10L 15/22; (...) (30) Unionist Priority: 07/02/2013 KR 102013-0013757 (73) Holder (s): SAMSUNG ELECTRONICS CO, LTD (72) Inventor (s): JIHYUN AHN; SORA KIM; JINYONG KIM; HYUNKYOUN KIM; HEEWOON KIM; YUMI AHN (74) Attorney (s): ORLANDO DE SOUZA (57) Summary: WIRELESS COMMUNICATION CHANNEL OPERATION METHOD AND PORTABLE TERMINAL SYSTEM A terminal enabled for the voice conversation function and a conversation control method are provided vocal, to provide distinct content based on the current emotional state, age and gender of the user. The mobile terminal that supports a voice conversation function includes a display unit, an audio processing unit, which selects content corresponding to a first criterion associated with a user in response to a user input, determines a content delivery scheme based on a second criterion associated with the user, and provides the selected content through the visualization unit and audio processing unit according to the content delivery scheme.
    110RADIOCOMUINATION
    120 ^ T
    CAMERA UNIT
    CONTROL UNIT
    SELECTION MODULE
    PROCESSING I! SUPPLY MODULE D
    I L 160 storage unit
    1/24
    WIRELESS COMMUNICATION CHANNEL OPERATION METHOD AND PORTABLE TERMINAL SYSTEM
    BACKGROUND OF THE INVENTION
    1. Field of the Invention [001] The present invention relates to a mobile terminal enabled for the voice conversation function and vocal conversation control method and, more particularly, to a terminal enabled for the voice conversation function and voice method. vocal conversation control to clearly produce content according to the user's current emotion, age and gender.
    2. Description of the Related Technique [002] The conventional vocal conversation function operates in such a way that an answer to a user's question is selected from a basic set of answers provided by the terminal manufacturer. Consequently, the vocal conversation function is limited by the fact that the same question is answered with the same answer regardless of the user. This means that when multiple users use the mobile terminal enabled for the voice conversation function, the conventional voice conversation function does not provide an optimized response per user.
    SUMMARY OF THE INVENTION [003] The present invention was made to solve at least the problems and inconveniences described above, and to provide at least the advantages described below. Consequently, an aspect of the present invention provides a mobile terminal for producing content that reflects the
    2/24 current emotional state, age and sex of a user, and a method of controlling their vocal conversation.
    [004] According to one aspect of the present invention, it is
    a terminal is provided mobile that supportsterminal includes a function inin conversation vocal. 0 an unity visualization, a processing unit in audio, and an
    control unit configured to select content matching the first criterion associated with a user in response to a user input, determine a content delivery scheme based on a second criterion associated with the user, and deliver the selected content through the display unit and audio processing unit according to the content delivery scheme.
    [005] In accordance with another aspect of the present invention, a method of vocal conversation from a mobile terminal is provided. The method includes selecting content corresponding to a first criterion associated with a user in response to a user input, determining a content delivery scheme based on a second criterion associated with the user, and providing the selected content through a display unit and of an audio processing unit of the mobile terminal according to the content delivery scheme.
    BRIEF DESCRIPTION OF THE DRAWINGS [006] The above and other aspects, features and advantages of modalities of the present invention will become evident from the detailed description below considered in conjunction with the accompanying drawings, in which:
    3/24 [007] FIG. 1 is a block diagram illustrating a configuration of the mobile terminal 100 according to an embodiment of the present invention;
    [008] FIG. 2 is a flow chart illustrating a method of controlling the vocal conversation function according to an embodiment of the present invention;
    [009] FIG. 3 is a table that maps emotional states and contents for use in the method of controlling vocal conversation according to a modality of the present invention;
    [010] FIGS. 4 and 5 are diagrams of screen displays that illustrate content delivery based on a first criterion according to an embodiment of the present invention;
    [011] FIG. 6 is a flow chart illustrating details of the acquisition step of the first criterion in FIG. 2;
    [012] FIG. 7 is a table that maps emotional states and contents for use in the method of controlling vocal conversation according to one embodiment of the present invention;
    [013] FIGS. 8 and 9 are diagrams of screen displays that illustrate content delivery based on the first criterion according to an embodiment of the present invention;
    [014] FIG. 10 is a table that maps emotional states and contents for use in the method of controlling the vocal conversation function according to an embodiment of the present invention;
    [015] FIG. 11 is a diagram of screen displays that illustrate content delivery based on the first
    Criterion according to an embodiment of the present invention;
    and [016] FIG. 12 is a schematic diagram illustrating a system for voice conversation function of the mobile terminal according to an embodiment of the present invention.
    DETAILED DESCRIPTION OF MODALITIES OF THE PRESENT INVENTION [017] The present invention will be described more fully hereinafter with reference to the accompanying drawings, in which illustrative embodiments of the invention are shown. This invention can, however, be embodied in many different forms and should not be considered as limited to the modalities presented here. Instead, these modalities are provided so that the description of this invention is thorough and complete, and will fully convey the scope of the invention to those skilled in the art. The present invention will be defined by the appended claims.
    [018] Although ordinal numbers are used, these terms are used only to distinguish one element, component, region, layer or section from another. Therefore, a first element, component, region, layer or section discussed below can be called a second element, component, region, layer or section without diverging from the teachings of the inventive concept. Descriptions should be understood as including any or all combinations of one or more of the items listed when the items are described using the term and / or similar.
    [019] FIG. 1 is a block diagram illustrating a configuration of the mobile terminal 100 according to an embodiment of the present invention.
    5/24 [020] With reference to FIG. 1, the mobile terminal 100 includes a radio communication unit 110, a camera unit 120, a location measurement unit 130, an audio processing unit 140, a display unit 160 and a control unit 170.
    [021] Radio unit 110 transmits / receives radio signals that carry data. Radio communication unit 110 may include a Radio Frequency (RF) transmitter configured to increase the frequency and amplify the transmission signals, and an RF receiver configured to amplify low noise and lower the frequency of the received signals. Radio communication unit 110 transfers data received by a radio channel to control unit 170 and transmits data produced by control unit 170 over the radio channel.
    [022] Camera unit 120 receives video signals. Camera unit 120 processes video frames of still and moving images obtained by an image sensor in video conference mode or image shoot mode. The camera unit 120 can supply the processed video frame to the display unit 150. The video frame processed by the camera unit 120 can be stored in the storage unit and / or transmitted externally via the radio communication unit 110.
    [023] Camera unit 120 may include two or more camera modules depending on the implementation of mobile terminal 100. For example, mobile terminal 100 may include a camera facing the same direction as the unit's screen
    6/24 viewing 150 and another camera facing the opposite direction of the screen.
    [024] The location measurement unit 130 may be provided with a satellite signal receiving module to measure the current location of the mobile terminal 100 based on the signals received from the satellites. By means of the radio communication unit 110, the location measuring unit 130 can also measure the current location of the mobile terminal 100 based on the signals received from the internal or external radio communication equipment within an installation.
    [025] The audio processing unit 140 may be provided with an encoder / decoder package that includes a data encoder / decoder for processing data packets and an audio encoder / decoder for processing the audio signal such as speech. The audio processing unit 140 can convert digital audio signals into analog audio signals via the audio encoder / decoder to provide the analog signal via a speaker (SPK) and convert the analog signal input through microphone (MIC) in digital audio signals.
    [026] Display unit 150 displays menus, input data, information on setting functions, etc. for the user in a visual mode. Display unit 150 displays a startup screen, a standby screen, a menu screen, a telephony screen, and other application execution screens.
    [027] The display unit 150 can be implemented with a Liquid Crystal Display (LCD), Organic Diodes
    7/24
    Light Emitters (OLED), Active Matrix OLED (AMOLED), flexible display, and a three-dimensional (3D) display.
    [028] The storage unit 160 stores programs and data necessary for the operation of the mobile terminal 100 and can be divided into a program region and a data region. The program region can store basic programs to control the entire operation of the mobile terminal 100, an Operating System (OS) to initialize the mobile terminal 100, applications for playing multimedia content, and other applications for performing optimal functions such as conversation vocal, camera, audio playback, and video playback. The data region can store the data generated in the state of use of the mobile terminal 100 such as still and moving images, phone book, and audio data.
    [029] The control unit 170 controls all operations of the components of the mobile terminal 100. The control unit 170 receives speech input from a user through the audio processing unit 140 and controls the display unit 150 to display the content corresponding to the user's speech in the vocal conversation function performed according to the user's manipulation. The control unit 170 can also reproduce the content corresponding to the user's speech via the audio processing unit 140. Here, the content can include at least one of the multimedia contents such as text, image, audio, film, and video clip. , and information such as weather conditions, recommended locations, and favorite contact.
    8/24 [030] In greater detail, the control unit 170 recognizes the user's speech to obtain the corresponding text. Then, the control unit 170 retrieves the content corresponding to the text and delivers the content through at least one of the display unit 150 and the audio processing unit 140. Finally, the control unit 170 can check the meaning of the text to retrieve the corresponding content among the related contents stored in the storage unit 160. In this way, using interactive voice communication, the user can be provided with a desired information through the stored related content. For example, if the user says Today's weather conditions the mobile terminal 100 receives the user's speech input through the audio processing unit 140. Then the mobile terminal 100 retrieves the content (information about weather conditions) corresponding to today's weather text obtained from the user's speech and provides the content retrieved via at least one of the display unit 150 and the audio processing unit 140.
    [031] In particular, in an embodiment of the present invention, the control unit 170 can select the content to be provided via the display unit and / or the audio processing unit 140 depending on the current emotion, age and sex of the user. For the purpose of accomplishing this, the control unit 170, according to an embodiment of the present invention, can include a module
    9/24 content selection 171 and a content delivery module 175.
    [032] FIG. 2 is a flow chart illustrating a method of controlling vocal conversation function according to an embodiment of the present invention.
    [033] With reference to FIG. 2, if the voice conversation function is performed in step S210, the content selection module 171 obtains a first criterion associated with the user in step S220. Here, the first criterion may include the user's current emotional state. The emotional state indicates a mood or sensation such as joy, sadness, anger, surprise, etc.
    [034] The content selection module 171 determines whether a user's speech input is detected in step S230. If a user's speech input is detected through the audio processing unit 140, the content selection module 171 selects the content corresponding to the user's speech input based on the first criterion in step S240. In more detail, the content selection module 171 obtains the phrase from the user's speech. Then, the content selection module 171 retrieves the contents corresponding to the sentence. Then, the content selection module 171 selects one of the contents using the predetermined emotional state information based on the first criterion. Here, information about specific content-emotional state can be pre-configured and stored in storage unit 160. The content selection module 171 can also retrieve content first based on the first criterion and on
    10/24 then select one of the contents corresponding to the sentences.
    [035] Otherwise, if no user speech input is detected in step S230, the content selection module 171 selects the content based on the first criterion in step S250.
    [036] If the content is selected, the content delivery module 175 obtains a second criterion associated with the user in step S260. Here, the second criterion can include at least one of the user's age and gender. The age of the user can be the exact age of the user or one of predetermined age groups. For example, the user's age can be indicated with an exact number such as 30 or 50, or with an age group such as 20's, 50's, child, adult, and elderly.
    [037] In detail, the content delivery module receives the user's face image from camera unit 120. The content delivery module 175 can obtain the second criterion automatically from the user's face image based on the information averages of faces of groups by age or sex stored in storage unit 160. The content delivery module 175 also receives the user's speech input through the audio processing unit 140. Then the content delivery module 175 can obtain the second criterion from the user's speech using the average information of group statements by age or sex. The content delivery module 175 can also obtain the second criterion based on the words that constitute the sentences obtained from the user's speech. At this point, the
    11/24 content delivery 175 can obtain the second criterion using the words of groups by age or by sex. For example, if a phrase I want new jellies is obtained from the user's speech, it is possible to judge the user as a child based on the word jellies.
    [038] The content delivery module 175 can obtain the second criterion based on both the face image and the user's speech. Although the description is directed to the case where the content delivery module 175 obtains the second criterion based on the user's face image and speech, the various modalities of the present invention are not limited to this one, but can be realized for the user introduce the second criterion. In this case, the entry of the second criterion by the user can be stored in the storage unit 160. The content delivery module 175 performs predetermined functions based on the second criterion stored in the storage unit 160.
    [039] If the second criterion is obtained, the content delivery module 175 determines a content delivery scheme based on the second criterion in step S270. That is, the content delivery module 175 determines the content delivery scheme by changing the words that make up the content selected by the content selection module 171, the delivery speed of the selected content and the output size of the selected content.
    [040] In greater detail, the content delivery module 175 can change the words that make up the selected content to words suitable for the second
    12/24 criteria based on word information by age groups or word information by sex. For example, if the content includes Pajamas Store and if the user belongs to the children age group, the content delivery module 175 changes the word Pajamas to the word Jim jams (jelly jelly) appropriate for children.
    [041] The content delivery module 175 determines the delivery speed of the selected content based on the delivery speed information of the age groups or the delivery speed information by sex stored in the storage unit 160. For example, if the user belongs to the child or elderly age group, the content delivery module 175 can decrease the speech reproduction speed of the selected content.
    [042] The content delivery module 175 also determines the output size of the selected content based on the groups output size information by
    age or in the size information of exit by sex. Per example, if the user belongs to group in age of old man, the supply module in content 175 can
    increase the output volume of the selected content and the display size (for example, font size) of the selected content based on the output size information of the age groups. Storage unit 160 stores a table that contains a mapping of the age or gender group to the content delivery scheme (speed and size of the content delivery), and the content delivery module 175 determines the content delivery scheme selected based on the data
    13/24 stored in the table mapping. If the content delivery scheme is selected, the content delivery module 175 delivers the content selected by the content selection module 171 via display unit 150 and audio processing unit 140 according to the content delivery scheme in step
    S280.
    [043] Then, if a request to end the voice conversation function is detected in step S290, the control unit 170 ends the voice conversation function. If a request to end the vocal conversation function is not detected in step S290, control unit 170 returns the procedure to step S220.
    [044] As described above, the vocal conversation control method of the invention selects content suitable for the user's current emotional state and determines the content delivery scheme according to the user's age and / or gender in order to provide customized content to the user. The method makes it possible to provide more realistic vocal conversation functionality.
    [045] However, if the phrase obtained from the user's speech input through the audio processing unit 140 is a request to change the content delivery scheme, the content delivery module 175 changes the provisioning scheme. content according to the phrase. For example, after the content has been delivered according to the content delivery scheme determined based on the second criterion, if the user says the phrase Can you speak faster and more quietly , the content delivery module 175
    14/24 increases speech reproduction speed by one degree and decreases audio volume by one degree.
    [046] The content delivery module 175 can store the changed content delivery scheme in storage unit 160. Then, the content delivery module 175 changes the content delivery scheme determined based on the second criterion using history delivery scheme for previously stored content. The content delivery module 175 can deliver the selected content according to the changed content delivery scheme.
    [047] A content delivery procedure according to an embodiment of the invention is hereinafter described with reference to FIGS. 3 to 5.
    [048] FIG. 3 is a table that maps emotional states and contents for use in the method of controlling vocal conversation according to a modality of the present invention. FIGS. 4 and 5 are diagrams of screen displays that illustrate content delivery based on the first criterion according to an embodiment of the present invention.
    [049] With reference to FIG. 3, the contents are premapped for emotional states. The emotional state of joy is mapped to content A, the emotional state of sadness to content B, the emotional state of anger to content C and surprise emotional state to content D. These emotional states and content are premapped and stored in storage unit 160 .
    [050] The content selection module 171 can select the appropriate content for the first criterion
    15/24 (current emotional state of the user) among contents by emotional state.
    [051] With reference to FIG. 4, based on the UT phrase obtained from the user's speech input through the audio processing unit 140 and the first criterion (current emotional state of the user), the content selection module 171 selects the content Ά (ATI) for the emotional state of joy and content B (AT2) for the emotional state of sadness.
    [052] With reference to FIG. 5, the content selection module 171 selects content C (ATI) for the emotional emotional state and content D (AT2) for the emotional surprise state, based on the first criterion (current emotional state of the user).
    [053] Although FIG. 3 is directed to a mapping of a content item by emotional state, the present invention is not limited to this, but it can be realized to map multiple content items by emotional state. In this case, the content selection module 171 can randomly select one of the multiple contents corresponding to the first criterion (current emotional state of the user).
    [054] The contents can be grouped by emotional state. A content group represents a set of content that have the same / similar property. For example, a content group can be classified into one of an action movie content group, an R&B music content group, etc. In this case, the content selection module 171 can select one of the contents of the
    16/24 content group that satisfies the first criterion (current emotional state of the user) randomly.
    [055] FIG. 6 is a flow chart illustrating details of the acquisition step of the first criterion in FIG. 2.
    [056] With reference to FIG. 6, the content selection module 171 obtains a user's face image from camera unit 120 in step S310 and detects the face area from the face image in step S320. That is, the content selection module 171 detects the face area having eyes, nose and mouth.
    [057] Next, the content selection module 171 extracts the fiducial points of the eyes, nose and mouth in step S330 and recognizes the facial expression based on the fiducial points in step S340. That is, the content selection module 171 recognizes the user's current expression based on the fiducial point information per expression stored in the storage unit 160.
    [058] Then, the content selection module 171 automatically retrieves the first criterion based on the expression determined based on the expression information by emotional state in step S350. Here, expression information by emotional state can be pre-configured and stored in storage unit 160.
    [059] Although the description is intended for the case where the content selection module 171 obtains the first criterion based on the image of the user's face, the present invention is not limited to this, but can be realized for the user to introduce the first criterion.
    17/24 [060] Another procedure for providing content according to an embodiment of the present invention is hereinafter described with reference to FIGS. 7 to 9.
    [061] FIG. 7 is a table that maps emotional states and contents for use in the method of controlling vocal conversation according to a modality of the present invention. FIGS. 8 and 9 are diagrams of screen displays that illustrate content delivery based on the first criterion according to an embodiment of the present invention.
    [062] The content selection module 171 can select content based on the first criterion (user's current emotional state) using the user's previous content playback history. The playback history of previous content is stored on storage unit 160 and is updated whenever the content is played according to the user's manipulation.
    [063] With reference to FIG. 7, the numbers of reproductions or respective content items are stored in storage unit 160. Content A1 is played three times, content A2 ten times, content BI five times, content B2 twice,
    contentcontent Cl eight times,DL twice the C2 content fifteenD2 one times, theturn. The and the content contents Al and A2 are mapped to the state emotional joy , the contents BI and B2 for the state emotional sadness , the contents Cl and C2 for the state emotional rage, and the contents DL and D2 for the state emotional surprise  (see FIG. 3). [064] 0 module content selection 171 may
    select one of the multiple content suitable for the
    18/24 first criterion (current emotional state of the user) based on the history of previous reproductions of the content.
    [065] With reference to FIG. 8, if the first criterion (current emotional state of the user) is joy, the content selection module 171 selects content A2 (ATI), which was most frequently reproduced from the contents Al and A2 mapped to the first criterion (state user's current emotional level). If the first criterion (current emotional state of the user) is sadness, the content selection module 171 selects the BI (AT2) content, which was most frequently reproduced from the BI and B2 content mapped to the first criterion (current emotional state) of user).
    [066] At this moment, the content selection module 171 can select the multiple content mapped to the first criterion (current emotional state of the user). Then the content delivery module 175 can determine the output positions of the multiple content based on the past reproduction history of the content.
    [067] With reference to FIG. 9, if the first criterion (current emotional state of the user) is joy, the content selection module 171 selects both content A1 and A2 as the content (ATI) that meets the first criterion (current emotional state of the user). Then the content delivery module 175 arranges the content Al under the content A2 (ATI) that has been reproduced most frequently. If the first criterion (current emotional state of the user) is sadness, the content selection module 171 selects both
    19/24 BI and B2 contents as contents (AT2) that satisfy the first criterion (current emotional state of the user). Then the content delivery module 175 places the B2 content under the BI (AT2) content that has been reproduced most often.
    [068] Another procedure for providing content according to an embodiment of the present invention is hereinafter described with reference to FIGS. 10 and 11.
    [069] FIG. 10 is a table that maps emotional states and contents for use in the vocal conversation control method according to a modality of the present invention. FIG. 11 is a diagram of screen displays that illustrate content delivery based on the first criterion according to an embodiment of the present invention.
    [070] The content selection module 171 can select content based on the first criterion (current emotional state of the user) and the history of providing content based on the user's previous emotional state. The content delivery history based on the user's previous emotional state is stored in storage unit 160 and is updated whenever the content is delivered according to the user's emotional state while the voice chat function is activated.
    [071] With reference to FIG. 10, the number of times of the previous exits based on the emotional state of the contents are stored in the storage unit
    160. Content Al was provided three times, content A2 eight times, content BI four times, content B2 one
    20/24 times, C1 content three times, C2 content eleven times, D1 content twice, and D2 content five times.
    [072] The content selection module 171 can select one of the multiple content mapped to the first criterion (current emotional state of the user) using the content delivery history based on the previous emotional state.
    [073] With reference to FIG. 11, if the first criterion (current emotional state of the user) is joy, the content selection module 171 selects the A2 content that was most frequently provided in association with the user's previous emotional state as the content (ATI) corresponding to the first criterion between contents A1 and A2. If the first criterion (user's current emotional state) is sadness, the content selection module 171 selects the BI content that was most often provided in association with the user's previous emotional state as the content (AT2) corresponding to the first criterion ( current emotional state of the user) between BI and B2 content.
    [074] The content selection module 171 can select all mapped content to satisfy the first criterion (current emotional state of the user). The content delivery module 175 then determines the exit positions of the multiple content using the content delivery history based on the previous emotional state. For example, if the first criterion (current emotional state of the user) is joy, the content selection module 171 selects both content A1 and A2 as the content
    21/24 corresponding to the first criterion (current emotional state of the user). Then the content delivery module 175 arranges the content A1 under the content A2 which was most frequently reproduced according to the user's previous emotional state.
    [075] Another procedure for providing content according to an embodiment of the present invention is hereinafter described.
    [076] The content selection module 171 can select content based on the first criterion (current emotional state of the user) using current location information from the mobile terminal 100 that is obtained through the location measurement unit 130. In greater detail, the content selection module 171 obtains multiple content based on the first criterion (current emotional state of the user). Then, the content selection module 171 selects the content associated with the area within a predetermined radius around the current location of the mobile terminal among the obtained contents. For example, if the content is information referring to recommended places (restaurant, cafe, etc.), the content selection module 171 can select the content suitable for the current location of the mobile terminal 100 based on the current location information of the terminal mobile.
    [077] Of course, the content selection module 171 can obtain multiple content associated with the area within a predetermined radius around the current location of the mobile terminal and then select the content that
    22/24 satisfies the first criterion (current emotional state of the user) among the contents obtained.
    [078] Although the description was intended for the case where the control unit 170, content selection module 171, and content delivery module 175 are configured separately and responsible for different functions, the present invention is not limited to this, but it can be realized in such a way that the control unit, the content selection module and the content delivery module work in an integrated mode.
    [079] FIG. 12 is a schematic diagram illustrating a system for voice conversation function of the mobile terminal according to an embodiment of the present invention.
    [080] Since the mobile terminal 100 here is identical to the mobile terminal described above with reference to FIG. 1, a detailed description of the mobile terminal 100 is omitted here. The mobile terminal 100 according to an embodiment of the present invention is connected to a server 200 via a wireless communication network 300.
    [081] In the modalities described above, the control unit 170 of the mobile terminal 100 performs the first criterion acquisition operation, the content selection operation based on the first criterion, the second criterion acquisition operation, and the determination operation content delivery scheme.
    [082] In this modality, however, the control unit 170 of the mobile terminal 100 exchanges data with the server through the radiocommunication unit 110, and performs the acquisition operation of the first criterion, the content selection operation based on the first criterion. , a
    23/24 operation to acquire the second criterion, and the operation to determine the content delivery scheme.
    [083] For example, the control unit 170 of the mobile terminal 100 provides the server 200 with the input of the user's face image through the camera unit 120 and the user's speech input via the audio processing unit 140. In then the server 200 obtains the first and second criteria based on the user's face image and the user's speech. The server 200 provides the mobile terminal 100 with the first and second criteria obtained.
    [084] Although the description was made under the consideration of a single user, the present invention is not limited to this, and can also be applied to the case where several users use the mobile terminal 100. In this case, it is necessary to add an operation to identify the current user of the mobile terminal 100. The user's previous content delivery scheme history, the user's previous content playback history and the content delivery history based on the user's previous emotional state can be stored per user . Consequently, even when multiple users use the mobile terminal 100, it is possible to provide user specific content.
    [085] As described above, the mobile terminal enabled for the vocal conversation function and the vocal conversation control method of the present invention are able to select any content suitable for the user's current emotional state and determine a content delivery scheme according to the user's age and gender. Consequently, it is possible to provide the contents
    24/24 customized for individual user. Consequently, the present invention is capable of implementing a realistic vocal conversation function.
    [086] Although embodiments of the invention have been described in detail above, a person skilled in the art will understand and understand that many variations and modifications of the basic inventive concept described here will still fall within the spirit and scope of the invention as defined in the following claims and their equivalent.
    1/6
    权利要求:
    Claims (27)
    [1]
    1. Mobile terminal that supports a voice conversation function, the terminal characterized by comprising:
    a display unit;
    an audio processing unit;
    a control unit configured to select content corresponding to a first criterion associated with a user in response to a user input, determine a content delivery scheme based on a second criterion associated with the user, and deliver the selected content through the visualization and audio processing unit according to the content delivery scheme.
    [2]
    2. Terminal, according to claim 1, characterized by the fact that the first criterion is a user's current emotional state, and the second criterion is user information that includes at least one among the user's age and sex.
    [3]
    3. Terminal, according to claim 1, characterized by the fact that the control unit selects the content corresponding to the first criterion, the corresponding content comprising at least one predetermined content according to the emotional state of the user.
    [4]
    4. Terminal, according to claim 1, characterized by the fact that the control unit selects the content based on the first criterion and the user's previous content playback history.
    [5]
    5. Terminal, according to claim 1, characterized by the fact that the control unit selects the
    2/6 content based on the first criterion and information on the current location of the terminal.
    [6]
    6. Terminal, according to claim 1, characterized by the fact that the control unit selects the content based on the history of content supply in association with the user's previous emotional states.
    [7]
    7. Terminal, according to claim 1, characterized by the fact that the audio processing unit receives speech from the user, and the unit and control select the content corresponding to a phrase obtained from the speech based on the first criterion.
    [8]
    8. Terminal, according to claim 7, characterized by the fact that the control unit obtains a second criterion based on the words that make up the sentence.
    [9]
    9. Terminal, according to claim 1, characterized by the fact that the control unit changes at least one of the words that make up the content, content output speed and content output size based on the second criterion and provide the content of according to the content delivery scheme.
    10. Terminal, from a deal with the claim 1, characterized by the fact of the unit processing in audio receive speech from the user, and the control unit change when a sentence obtained from speech is an request to change the scheme supply in content, the delivery scheme of content. 11. Terminal, from a deal with the claim 1,
    characterized by the fact that the control unit changes the content delivery scheme determined based on the
    3/6
    second criterion using history of scheme provision of the user's previous content and provide the content according with the scheme supply content changed. 12. Terminal, according to the claim 1,
    characterized by also comprising a camera unit that captures a user's face image, where the control unit automatically obtains the first criterion based on the user's face image.
    [10]
    13. Terminal, according to claim 12, characterized by the fact that the control unit obtains the first criterion from predetermined information of expressions by emotional state based on the facial expressions obtained from the image of the user's face.
    [11]
    14. Terminal, according to claim 1, characterized by also comprising a camera unit that obtains an image of the user's face, where the audio processing unit receives speech from the user and the control unit automatically obtains the second criterion with based on at least one of the user's face images and speech.
    [12]
    15. Terminal, according to claim 1, characterized by the fact that the control unit receives the first and second criteria through the audio processing unit.
    [13]
    16. Voice conversation method of a mobile terminal, the method characterized by comprising:
    select content corresponding to a first criterion associated with a user in response to a user input;
    4/6 determine a content delivery scheme based on a second criterion associated with the user; and providing the selected content via a display unit and a mobile terminal audio processing unit according to the content delivery scheme.
    [14]
    17. Method, according to claim 16, characterized by the fact that the first criterion is a current emotional state of the user, and the second criterion is information about the user including at least one among the user's age and sex.
    [15]
    18. Method, according to claim 16, characterized by the fact that the selection of the content comprises selecting the content corresponding to the first criterion, the corresponding content comprising at least one predetermined content according to the emotional state of the user.
    [16]
    19. Method, according to claim 16, characterized by the fact that the content selection comprises selecting the content based on the first criterion and the user's previous content playback history.
    [17]
    20. Method, according to claim 16, characterized by the fact that the selection of the content comprises selecting the content based on the first criterion and on the current location information of the terminal.
    [18]
    21. Method, according to claim 16, characterized by the fact that the content selection comprises selecting the content based on the history of content delivery in association with the user's previous emotional states.
    5/6
    [19]
    22. Method, according to claim 16, characterized by still understanding to receive speech from the user, where the selection of the content comprises selecting the content corresponding to a phrase obtained from the speech based on the first criterion.
    [20]
    23. Method according to claim 22, characterized by further comprising the acquisition of a second criterion based on the words that make up the sentence.
    [21]
    24. Method, according to claim 16, characterized in that the determination of the content delivery scheme comprises changing at least one of the words that make up the content, speed of the content's output and size of the content's output based on the second criterion , and deliver the content according to the content delivery scheme.
    [22]
    25. Method, according to claim 24, characterized in that it also comprises receiving speech from the user, and where the determination of the content delivery scheme comprises changing, when a phrase obtained from the speech is a request to change the delivery scheme content, the content delivery scheme.
    [23]
    26. Method according to claim 16, characterized in that the determination of the content delivery scheme comprises changing the determined content delivery scheme based on the second criterion using the user's previous content delivery scheme history.
    6/6
    [24]
    27. The method of claim 16, further comprising:
    receive an image of the user's face; and automatically obtain the first criterion based on the user's face image.
    [25]
    28. Method, according to claim 27, characterized by the fact that the acquisition of the first criterion comprises the acquisition of the first criterion from predetermined information of expressions by emotional state based on the facial expressions obtained from the image of the user's face.
    [26]
    29. The method of claim 16, further comprising:
    receive at least one of an image of the user's face and speech; and automatically obtain the second criterion based on at least one of an image of the user's face and speech.
    [27]
    30. Method according to claim 16, characterized in that it further comprises receiving the first and second criteria through the audio processing unit.
    1/12
    类似技术:
    公开号 | 公开日 | 专利标题
    BR102014003021A2|2018-04-10|WIRELESS COMMUNICATION CHANNEL OPERATION METHOD AND PORTABLE TERMINAL SYSTEM
    CN106465074B|2020-01-31|Use of digital assistant in communication
    US11107466B2|2021-08-31|Digital assistant voice input integration
    US9164984B2|2015-10-20|Delay in video for language translation
    US9686627B2|2017-06-20|Multidimensional virtual learning system and method
    BRPI0706212A2|2011-03-15|caller classification in a network-based conference
    US9560316B1|2017-01-31|Indicating sound quality during a conference
    US20150180919A1|2015-06-25|Active talker activated conference pointers
    US10250846B2|2019-04-02|Systems and methods for improved video call handling
    WO2017222645A1|2017-12-28|Crowd-sourced media playback adjustment
    US20210390148A1|2021-12-16|Configuring output controls on a per-online identity and/or a per-online resource basis
    US11272051B2|2022-03-08|Method for notification reminder, terminal, and storage medium
    CN110989889A|2020-04-10|Information display method, information display device and electronic equipment
    WO2018058211A1|2018-04-05|System and method for managing mobile telephone calls
    CN111225318A|2020-06-02|Audio adjusting method and device and electronic equipment
    TWM426227U|2012-04-01|Communication device with ringtone function
    同族专利:
    公开号 | 公开日
    CN103984408A|2014-08-13|
    TWI628650B|2018-07-01|
    JP2014153715A|2014-08-25|
    KR102050897B1|2019-12-02|
    JP6541934B2|2019-07-10|
    RU2661791C2|2018-07-19|
    US20140222432A1|2014-08-07|
    CA2842005A1|2014-08-07|
    KR20140100704A|2014-08-18|
    AU2014200660B2|2019-05-16|
    EP2765762A1|2014-08-13|
    TW201435857A|2014-09-16|
    AU2014200660A1|2014-08-21|
    RU2014104373A|2015-08-20|
    EP2765762B1|2019-07-10|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    JPH08255150A|1995-03-17|1996-10-01|Toshiba Corp|Information public offering device and multimodal information input/output system|
    JPH10326176A|1997-05-23|1998-12-08|Oki Hokuriku Syst Kaihatsu:Kk|Voice conversation control method|
    JP2001215993A|2000-01-31|2001-08-10|Sony Corp|Device and method for interactive processing and recording medium|
    WO2002034478A1|2000-10-23|2002-05-02|Sony Corporation|Legged robot, legged robot behavior control method, and storage medium|
    US6964023B2|2001-02-05|2005-11-08|International Business Machines Corporation|System and method for multi-modal focus detection, referential ambiguity resolution and mood classification using multi-modal input|
    JP2003046980A|2001-08-02|2003-02-14|Matsushita Electric Ind Co Ltd|Method, device, and program for responding to request|
    US9374451B2|2002-02-04|2016-06-21|Nokia Technologies Oy|System and method for multimodal short-cuts to digital services|
    JP2004310034A|2003-03-24|2004-11-04|Matsushita Electric Works Ltd|Interactive agent system|
    JP2005065252A|2003-07-29|2005-03-10|Fuji Photo Film Co Ltd|Cell phone|
    US7881934B2|2003-09-12|2011-02-01|Toyota Infotechnology Center Co., Ltd.|Method and system for adjusting the voice prompt of an interactive system based upon the user's state|
    JP2005157494A|2003-11-20|2005-06-16|Aruze Corp|Conversation control apparatus and conversation control method|
    JP2005275601A|2004-03-23|2005-10-06|Fujitsu Ltd|Information retrieval system with voice|
    JP2006048663A|2004-06-30|2006-02-16|Metallic House Inc|System and method for order receiving and ordering article/service, server device and terminal|
    JP2006146630A|2004-11-22|2006-06-08|Sony Corp|Content selection reproduction device, content selection reproduction method, content distribution system and content retrieval system|
    US8214214B2|2004-12-03|2012-07-03|Phoenix Solutions, Inc.|Emotion detection device and method for use in distributed systems|
    BRPI0607359B1|2005-02-04|2019-04-02|Qualcomm Incorporated|SAFE SELF-INITIATION FOR WIRELESS COMMUNICATIONS|
    US7490042B2|2005-03-29|2009-02-10|International Business Machines Corporation|Methods and apparatus for adapting output speech in accordance with context of communication|
    US7672931B2|2005-06-30|2010-03-02|Microsoft Corporation|Searching for content using voice search queries|
    US20070288898A1|2006-06-09|2007-12-13|Sony Ericsson Mobile Communications Ab|Methods, electronic devices, and computer program products for setting a feature of an electronic device based on at least one user characteristic|
    KR20090085376A|2008-02-04|2009-08-07|삼성전자주식회사|Service method and apparatus for using speech synthesis of text message|
    JP2010057050A|2008-08-29|2010-03-11|Sharp Corp|Information terminal device, information distribution device, information distribution system, and program|
    AT557388T|2008-12-19|2012-05-15|Koninkl Philips Electronics Nv|METHOD AND SYSTEM FOR ADAPTING COMMUNICATION|
    US8340974B2|2008-12-30|2012-12-25|Motorola Mobility Llc|Device, system and method for providing targeted advertisements and content based on user speech data|
    JP2010181461A|2009-02-03|2010-08-19|Olympus Corp|Digital photograph frame, information processing system, program, and information storage medium|
    KR101625668B1|2009-04-20|2016-05-30|삼성전자 주식회사|Electronic apparatus and voice recognition method for electronic apparatus|
    US10540976B2|2009-06-05|2020-01-21|Apple Inc.|Contextual voice commands|
    CN102447836A|2009-06-16|2012-05-09|英特尔公司|Camera applications in a handheld device|
    US20120011477A1|2010-07-12|2012-01-12|Nokia Corporation|User interfaces|
    KR101916107B1|2011-12-18|2018-11-09|인포뱅크 주식회사|Communication Terminal and Information Processing Method Thereof|
    CN102541259A|2011-12-26|2012-07-04|鸿富锦精密工业（深圳）有限公司|Electronic equipment and method for same to provide mood service according to facial expression|EP2967322A4|2013-03-11|2017-02-08|Magic Leap, Inc.|System and method for augmented and virtual reality|
    CN113918018A|2013-03-15|2022-01-11|奇跃公司|Display system and method|
    KR102340251B1|2014-06-27|2021-12-16|삼성전자주식회사|Method for managing data and an electronic device thereof|
    JP6596865B2|2015-03-23|2019-10-30|日本電気株式会社|Telephone, telephone system, telephone volume setting method, and program|
    JP6601069B2|2015-09-01|2019-11-06|カシオ計算機株式会社|Dialog control apparatus, dialog control method, and program|
    KR20170034154A|2015-09-18|2017-03-28|삼성전자주식회사|Method and electronic device for providing contents|
    CN105700682A|2016-01-08|2016-06-22|北京乐驾科技有限公司|Intelligent gender and emotion recognition detection system and method based on vision and voice|
    JP6927989B2|2016-02-25|2021-09-01|コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ．|Communication equipment and methods for determining call priority level and / or conversation duration|
    US10638091B2|2016-07-28|2020-04-28|Sony Corporation|Information processing device and information processing method|
    US10276149B1|2016-12-21|2019-04-30|Amazon Technologies, Inc.|Dynamic text-to-speech output|
    CN106873800A|2017-02-20|2017-06-20|北京百度网讯科技有限公司|Information output method and device|
    US20180350371A1|2017-05-31|2018-12-06|LenovoPte. Ltd.|Adjust output settings based on an identified user|
    US10983753B2|2017-06-09|2021-04-20|International Business Machines Corporation|Cognitive and interactive sensor based smart home solution|
    JP2019008113A|2017-06-23|2019-01-17|カシオ計算機株式会社|Electronic apparatus, emotion information acquisition system, program and emotion information acquisition method|
    CN108920129A|2018-07-27|2018-11-30|联想有限公司|Information processing method and information processing system|
    EP3844559A4|2018-08-31|2021-10-27|Magic Leap, Inc.|Spatially-resolved dynamic dimming for augmented reality device|
    CN109637519B|2018-11-13|2020-01-21|百度在线网络技术（北京）有限公司|Voice interaction implementation method and device, computer equipment and storage medium|
    JP6533634B1|2018-12-25|2019-06-19|クックパッド株式会社|Server apparatus, information processing terminal, system, method and program|
    KR20200111853A|2019-03-19|2020-10-05|삼성전자주식회사|Electronic device and method for providing voice recognition control thereof|
    CN113380240A|2021-05-07|2021-09-10|荣耀终端有限公司|Voice interaction method and electronic equipment|
    法律状态:
    2018-04-10| B03A| Publication of a patent application or of a certificate of addition of invention [chapter 3.1 patent gazette]|
    2018-04-17| B08F| Application dismissed because of non-payment of annual fees [chapter 8.6 patent gazette]|
    2018-08-07| B08K| Patent lapsed as no evidence of payment of the annual fee has been furnished to inpi [chapter 8.11 patent gazette]|
    优先权:
    申请号 | 申请日 | 专利标题
    KR1020130013757A|KR102050897B1|2013-02-07|2013-02-07|Mobile terminal comprising voice communication function and voice communication method thereof|
    KR10-2013-0013757|2013-02-07|
    [返回顶部]