lyrisys: an interactive support system for writing lyrics based on topic transition
TRANSCRIPT
LyriSys: An Interactive Support System
for Writing Lyrics Based on Topic TransitionKento Watanabe1
Yuichiroh Matsubayashi1, Kentaro Inui1, Tomoyasu Nakano2, Satoru Fukayama2, Masataka Goto2
1Graduate School of Information Sciences, Tohoku University, Japan 2National Institute of Advanced Industrial Science and Technology (AIST), Japan
Difficulty of Writing Lyrics
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 2
It is difficult to consider many techniques at the same time.
Select words whose syllables correspond to the melody notes.
[Austin+2010, Ueda 2010]
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
I re-mem-ber our love And I re-mem-ber eve- ry word un- spo- ken1 3 1 1 1 1 3 2 1 3Syllable Counts
Lyrics
Difficulty of Writing Lyrics
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 3
It is difficult to consider many techniques at the same time.
[Austin+2010, Ueda 2010]
Block1 Block2 Block3I drop into the oceanMy friend the stormy seaAs long as the blindnessWon't leave me tonight
I felt the wrath of the chosen oneI wanted forgiveness but was given noneLet all the lights scream in my eyesThat 's how I want it to fee
I fell in loveI fell in love
Topic:Dark
Topic:Scene
Topic:Sweet Love
• Verse-Bridge-Chorus segment (called Block).• Text in each block has an topic.• The topic transition constitutes a story.
Development of a Lyrics Writing Support System
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 4
System enables rich interaction by the topic transition model
Topic Transition Model
Language Model
Sweet love
Scene
Life
Exciting
Ardent Love
……ni
ght
day
light …love
know
baby … pr
obab
ility
wordword
…
prob
abili
ty
SYSTEM INTELIGENCE
Over 100000 lyricsI drop into the oceanMy friend the stormy seaAs long as the blindnessWon't leave me tonight…
TRAINING DATA
USER INPUT
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
Syllable counts
e.g., “Time”→”Life”→”Love”
Number of blocksNumber of lines
Structure of an entire song…
Sequence of topicse.g.,1-3-1-1
SYSTEM OUTPUTRecommended lyrics
If tomorrow never comesI remember the dayYou realize time flies
…
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
…
Unsupervised Learning
Trial-and-Error
System Introduction
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 5
Example of Created Lyrics
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 6
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . . 1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .If you rea- lly un- der- stand I pro- mise you’-ll be An ex- traor- di- nar- y man
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
If you really understand I promise you'll be An extraordinary man
I don't really understand I promise you'll be Oh supernatural love
Example of lyrics when the user uses our system
Fully automatically recommended lyrics
The love phrases “I promise…” and ”supernatural love” were recommended when the topic was ⟨sweet love⟩.
1 1 2 3 1 2 2 1 1 5 1SyllableLyrics
Inputted Topic: Sweet Love
Writing Support System: LyriSys
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 7
Writing Support System: LyriSys
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 8
Writing Support System: LyriSys
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 9
Writing Support System: LyriSys
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 10
Writing Support System: LyriSys
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 11
Writing Support System: LyriSys
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 12
Writing Support System: LyriSys
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 13
Writing Support System: LyriSys
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 14
Writing Support System: LyriSys
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 15
Implementation of LyriSys
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 16
Recommendation Algorithm
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 17
LyriSys searches the word strings so that the word probability is large according to the beam search.
IYou
So…
1 syllable words
need
don’twhen…
1 syllable words
always
reallybetter…
2 syllable words
remember
everything
understand…
3 syllable words
: Word probability 𝑃(𝑤𝑜𝑟𝑑)|𝑤𝑜𝑟𝑑)+,, 𝑡𝑜𝑝𝑖𝑐)
Example of beam search when syllable count is “1-1-2-3” and topic is “sweet love”.
How does the system handle topic
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 18
Block1 Block2 Block3I drop into the oceanMy friend the stormy seaAs long as the blindnessWon't leave me tonight
I felt the wrath of the chosen oneI wanted forgiveness but was given noneLet all the lights scream in my eyesThat 's how I want it to fee
I fell in loveI fell in love
Topic:Dark
Topic:Scene
Topic:Sweet Love
𝑃(𝑡𝑜𝑝𝑖𝑐3|𝑡𝑜𝑝𝑖𝑐,) 𝑃(𝑡𝑜𝑝𝑖𝑐4|𝑡𝑜𝑝𝑖𝑐3)
∏𝑃(𝑤𝑜𝑟𝑑|𝑡𝑜𝑝𝑖𝑐,) ∏𝑃(𝑤𝑜𝑟𝑑|𝑡𝑜𝑝𝑖𝑐3) ∏𝑃(𝑤𝑜𝑟𝑑|𝑡𝑜𝑝𝑖𝑐4)
The story is represented by two kinds of probabilities
Transition probability
Word probability
Enhanced Hidden Markov Model [watanabe+ 2014]
How to Get Probabilities ?
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 19
𝑃(𝑤𝑜𝑟𝑑|𝑡𝑜𝑝𝑖𝑐)𝑃(𝑡𝑜𝑝𝑖𝑐)|𝑡𝑜𝑝𝑖𝑐)+,)Enhanced HMM [Watanabe+2014]
(1) Transition prob: (2) Word prob:
……
nigh
tda
ylig
ht …love
know
baby …
prob
abili
ty
wordword
…
SceneExcitingSlangTime
Sweet love
LifeForeign
DarkReligious
Ardent love
Scen
eEx
citin
gSl
ang
Tim
e
Swee
t lov
e
Life
Fore
ign
Dar
kRe
ligio
us
Ard
ent l
ove
Transition Matrix
𝑡𝑜𝑝𝑖𝑐)
𝑡𝑜𝑝𝑖𝑐 )+,
Prev
ious
73 5 4 1 7 4 5 1 1 1
6 62 3 1 15 1 5 1 5 1
9 5 42 6 8 9 13 1 5 1
1 1 3 85 1 5 3 1 2 1
6 9 2 1 72 1 3 1 7 1
8 1 6 8 1 71 4 1 1 16 5 5 2 7 2 67 1 4 1
1 1 0 1 1 1 1 93 1 1
2 6 2 2 13 0 4 0 69 1
2 4 2 3 4 1 3 0 4 78 (%)
System automatically learns (1) typical topic transitions and (2) semantically appropriate wordings
from a large collection of human lyrics.
prob
abili
ty
Over 10000 lyrics
Unsupervised learning
I drop into the oceanMy friend the stormy seaAs long as the blindnessWon't leave me tonight…
User Feedback
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 20
User Feedback: SetupnWriting Japanese Lyrics
p5 Japanese (One user was a school teacher of music).pWe randomly selected songs from RWC music database.
n4 Tasks1. Base line : Without interface2. Method 1: Fully Automatic lyrics generation
Users can only select topics and input syllables,cannot edit/select the generated lyrics.
3. Method 2: Interaction without topicSystem calculates the simple N-gram probability,cannot handle topic.
4. Proposed : LyriSys2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 21
User Feedback: Results
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 22
Task Positive Comments Negative Comments(Base line)Without system
In comparison to other tasks, it was comfortable in not specifying the number of syllable.
It was difficult to come up with the words that satisfy the melody of song, because of lacking in vocabulary.
(Method 1)Automatically generation
It was easy to write the lyrics because I didn’t need to determine which words to use.
I sometimes felt boring because users couldn’t edit the generated lyrics.
(Method 2)Interactionwithout topic
It was useful to select the candidate of lyrics when the generated result was partially good.
It was difficult to write the lyrics that represent the story, because only a limited variety of words are generated.
(Proposed)LyriSys
1) In comparison to the previous method 2, selecting topics made it easy to write the lyrics that specifies my intention. 2) The generated lyrics are more expressive than the result of other interface because of the consideration of topic.
1) The list of the 10 topics was too restricted and coarse-grained. 2) Although the system generates an abstract story, I thought that it would be interesting if the system could generate a concrete story.
Conclusions and Future Work
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 23
Our interaction designSystem is designed as a recommendation system.User can input the structure of entire an song and the story.
Our system intelligenceSystem automatically learns typical topic transitions and semantically appropriate wordings from a large collection of human lyrics.
Novel lyric-writing system: LyriSys
It might be too much of a burden for the user to specify the number of syllable counts.We plan to introduce extended functions on the Web.
Next Step
Appendix
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 25
Related worksMany researchers have studied rhyme of lyrics
[Nichols+09 Genzel+10, Abe+10 Ramakrishnan+10, Berbieri+12].
Prior studies report systems that can generate only a single line of lyrics independently of the rest [Abe+2012]
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 26
Our system has a strong advantage in capturing topic transition (i.e., story) of an entire lyrics.
Story Estimation
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 27
What topics were learned ?
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 28
z Label Representative words in each topic: top words from 𝑃(𝑤𝑜𝑟𝑑|𝑧)1 Sweet love love,know,baby,want,need,make,wan,feel,one,tell,give,more,heart,good,only,hold,'cause,please,
kiss,mine,stay,true,cry,crazy,touch,someone,miss,somebody,mean,woman,nobody,enough,care
2 scene night,day,light,eye,fall,dream,sun,sky,rain,shine,fly,into,home,star,walk,blue,wind,burn,fire,moon,christmas,cold,morning,watch,sleep,tear,dark,water,close,summer,open,remember,river,
3 exciting yeah,down,come,hey,gon,ooh,back,rock,keep,right,dance,tonight,stop,here,run,roll,alright,whoa,everybody,round,turn,ready,move,music,ride,party,bring,doo,c'mon,slow,beat,high,gim,ohh
4 slang nigga,shit,fuck,bitch,cause,money,niggaz,'em,ass,hit,real,hoe,game,wit,big,street,fuckin,bout,gettin,rap,hood,gun,block,motherfucker,dick,check,thug,catch,smoke,tryin,young,throw,straight
5 time have,all,time,say,never,see,way,take,think,life,find,try,thing,leave,too,look,nothing,lose,believe,live,always,everything,mind,change,long,something,wait,much,break,end,wrong,word,inside
6 Ardent love get,like,girl,off,body,put,boy,shake,hot,show,work,club,floor,lady,drop,cuz,sexy,got,sex,jump,low,lookin,freak,top,shawty,damn,pop,aint,boom,bounce,chick,dont,hair,thang,lil,shorty,push
7 life out,well,little,call,old,play,friend,talk,new,town,bad,use,car,kid,mama,drive,sit,lot,door,pay,drink,meet,pretty,house,buy,first,daddy,fun,guy,wear,next,write,phone,bed,school,hang,trouble,blues
8 foreign que,por,con,amor,como,una,quiero,para,sin,esta,pero,todo,solo,las,cuando,hay,soy,corazon,voy,vida,del,porque,los,tengo,bien,ella,estoy,ser,vez,hoy,aqui,les,este,puedo,siempre,dale,tan,quien
9 dark die,hand,head,lie,blood,face,dead,kill,fight,death,fear,hell,black,while,hate,wall,line,cut,scream,skin,bleed,pull,speak,become,bone,devil,full,thought,sick,blind,human,chain,build,stone,breath,
10 religious dem,god,man,lord,yuh,sing,nah,world,jesus,king,child,nuh,hear,gal,name,fus,song,praise,people,heaven,bear,inna,soul,pray,holy,war,free,pon,peace,hallelujah,glory,joy,thank,bless,mary,mek
Enhanced HMM
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 29
Generation process
For each topic z = 1,… , 𝐽:Draw 𝜃>→~𝐷𝑖𝑟 𝛼Draw 𝜙>~𝐷𝑖𝑟(𝛽)
For each lyrics 𝑚 = 1,2, … ,𝑀:For each Block 𝑏 = 1,2, … , 𝐵J:
Draw 𝑧K~𝑀𝑢𝑙𝑡𝑖 𝜃>NOP→For each word 𝑤 in block 𝑏:
Draw 𝑤~𝑀𝑢𝑙𝑡𝑖(𝜙>N)𝑤
𝑁,
𝑀
𝑧,
𝑁3 𝑁4𝑤
𝑧3
𝑤
𝑧4
𝜙>𝐽
𝜃>→𝐽
𝛽
𝛼
Plate notation of enhanced HMM
𝑃 𝐿𝑦𝑟𝑖𝑐𝑠J =UV𝜃>NOP→V𝜙>NWNX
YN
Z[,
\]
K[,
�
>_``
Generation probability of lyrics
topic
Transition distribution
Word distribution
DatasetnPurchasable at online shop.
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 30
$260
Development of a Lyrics Writing Support System
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 31
Our interaction design
System enables rich interaction by the topic transition model
System is designed as a recommendation system.
USER INPUT SYSTEM OUTPUT
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
Syllable counts
e.g., “Time”→”Life”→”Love”
Number of blocksNumber of lines
Structure of an entire song…
Sequence of topics
e.g., 1-3-1-1
Trial-and-Error
Recommended lyrics
If tomorrow never comesI remember the dayYou realize time flies
…
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
1
hogeGlassy FM Lead=126.
1
& \\17
& D . . . . . . D . . . . . . . . E .". . . D E ." . . . .
24
& . . . . . . . . # ." . # .! . # .! - D E ." . .
. . D E ." . .. .
30
& . . . . . . . . D E ." . . .. . . . . . . . . . . . D . .
35
& . . . . . . . . . . E ." .. . . . . . . . . . . . . . . . .
40
& . . . . . . . . . . . . . . . . . . . . . . . . # . . -44
& . . . . . . . . . . D E ." . . . . . . . . . . .49
& E ." . . . . . . . . . . . . . . . . . . . . E ." . . . . . . . . D54
& D E ." . . . . . . . . . . . . . . D . . . . . . . . .
58
& . . . . . . . . . . - D . . . . .
…
Development of a Lyrics Writing Support System
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 32
Our system intelligence
System enables rich interaction by the topic transition model
System automatically learns typical topic transitions and semantically appropriate wordings from a large collection of human lyrics.
Topic Transition Model
Language Model
Sweet love
Scene
Life
Exciting
Ardent Love
……
nigh
tda
ylig
ht …love
know
baby …
prob
abili
ty
wordword
…pr
obab
ility
Over 100000 lyricsI drop into the oceanMy friend the stormy seaAs long as the blindnessWon't leave me tonight…
Unsupervised Learning
SYSTEM INTELIGENCE
TRAINING DATA
Result: Automatic lyrics generation
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 33
# Block1: timeI know everything isAnd I remember every word unspokenRemember waiting there inside my headConversation circles
# Block2: timeI know everything isAnd I remember every word every thoughtTomorrow is another story about loveThe situation became
# Block3: lifeAnd I remember everythingCalifornia face realityAnd you realize you're lookingYou'll always rememberAnd you'll always beAnd everybody knows
# Block4: ardent loveI know you're watching every dreamI really don't want nobody sitting on topAnd I remember feeling like letting goI really don't matter
# Block5: sweet loveI remember DecemberI promise you'll remember the first degreeI don't really understandI promise you'll beOh supernatural love
Result: User interaction with system
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 34
# Block1: timeI remember our loveAnd I remember every word unspokenRemember waiting there inside my headRemember I believe
# Block2: timeI know everything endsBut I remember every word every thoughtTomorrow is another story about loveSo appreciate it all
# Block3: lifeYou can remember yesterdayReminiscing on everyoneAnd you realize you're lookingYou'll always rememberAnd every single dayYou probably say
# Block4: ardent loveIt's all because you're wasting timeBut you're the one you're looking out forAnd I remember feeling like letting go'Cause I don't need anybody
# Block5: sweet loveI remember DecemberAnd every little tenderness, and I believeIf you really understandI promise you'll beAn extraordinary man
This result shows that the created lyrics correspond to the input parameters (i.e., syllables and topics)
We can see the sentimental phrases “in my mind” and ”I just wanna be with you” were created when the topic was ⟨Sweet Love⟩.
Example 2 (made by user)
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 35
世界中でこんなにすれ違い.涙を見せず独り振る舞うだけで.あなたのそばに.(We could not understand each other in this world. I didn’t show my tears, I was lonely. I just wanna be with you.)
世界中でこんなにたくさんの.心に決めて思い出した思い出に.あなたのそばに.(So many things in the world. Memories that I remembered in my mind. I just wanna be with you.)
Block 2, Topic: Sweet Love
se- ka- i- ju- u de ko-n- na- ni su-re-chi-ga- i na-mi- da wo mi-se-zu hi-to-ri hu-ru-ma-u da-ke de a- na- ta no so- ba ni
Fully automatically generated lyrics.
Example outcome of an user’s interactions with the system.
This result shows that the created lyrics correspond to the input parameters (i.e., syllables and topics)
We can see the scenic phrases “the way of memories” and ”wet in the rain” were created when the topic was ⟨Scene⟩.
Example (made by user)
2017/3/16 The 22nd annual meeting of the intelligent user interfaces community (IUI2017) 36
思い出の坂道は久しぶりの昼下がりへ.恋人の足跡はアスファルトの雨に濡れる.
o-mo-i- de no sa-ka-mi-chi wa hi- sa-si-bu-ri no hi- ru sa-ga-ri e ko- i- bi-to no a- shi-a- to ha a- su-fa-ru-to no a- me ni nu-re-ru
(Afternoon came to the way of memories after a long time. Lovers' footprint get wet in the rain on asphalt.)
思い出の坂道を.雨上がりの交差点で.思い出の坂道を.心にない雨に濡れて.(The way of memories. At the intersection of the rain. The way of memories. I got wet in the rain without heart.)
Block 1, Topic: Scene
Fully automatically generated lyrics.
Example outcome of an user’s interactions with the system.