今更感がかなりあるものの、、RNN: Recurrent Neural Networkって面白そうだなと思い、CharRNNのソースに触れてみた。
オリジナルはこちら。chainerバージョンです(chainer-char-rnn)。有り難く使わせて頂きました。
github.com
いきなりですが調査結果
- 正直CPUでもGPUでもスピードにそんなに違いはない。やはりそこがRNNの課題点?
- CharRNNはpython2で組まれてるので注意。
- 実行環境はwindows/x64/GPU/Python3.5(3.5用に少し改変)
テスト実行にあたってのログ
Train:(CPU)
$python train.py
Train:(GPU)
$python train.py --gpu 0
- 学習ではCVフォルダが生成され、その中にチェックポイントでのモデルファイルがセーブされていく
Test:(CPU):
- vocalファイルと、モデル、最初の一字を少なくとも指定する(word-seqモデルなので最初の1字だけ指定すればいい)
$python sample.py --vocabulary data/tinyshakespeare/vocab.bin --model cv/charrnn_epoch_0.04.chainermodel --primetext hello
すると最初に生成されたモデルはこんな感じ
helloa oaeTa
nehnreyttffpmywa e pree
n nlg
ds e.
qocs,oo eaAoaopdmrh toslpyiootta aoeRler eaGehoate Cfass dmome et, oo
iDN eoMPzdlnhtokTrn nor Mo:huramth dsdlannhoroh Rhourk,lsrteeos hnhioWo aediciunsordGhosadrh
n,e
e. nD I;s
oer s hlr Topi
P:sfTem, nyetbcki
rJ e s p eiefto pilishrab.ybnAegtI.vspiyir .eoo od en'oNo
ifN reeki?aRaleeehe LLny e l,a;e s troeotd uo oaSo uh ,nh gdha nhdirC;
e n mn ' lra
rre;cew tek
b, OD
,
es etgspnee. iMSCcsslnag iiaM
:dL, kttw ae Zrdih rcyar n hswEM FDgbnh emsl
:eNo.';Ced
j:odnqs t h
iEaraetoyd ffaehpiokot
hs,tfe ciuf.oith
eCUobuS?EANLSINrmgeonesfndoiiaa;lnpn esdtnbit IWtTho'ledc tn egna Ga sNbci h.:e ssght duhtT oobdo
hsig
ee,ehnmr:
bM
s
em
IX y?Te,ilruq
pcdtlwratiohmsda ! r
u eolu fhw
pholNIm e d usho,ewoOhotl dL ..n tUlLwsrCgbkte a
esoee eoimRo Dnyan br
mertm- aryev Ifed tg
h
dcEu i ne,hrgfoec.a-ymradsmwso: iew uuas
ohSenudoettrrmerl tbsa'dk dan.yvh
f
iontsniEeereii ,nityyueeheyoerslnqe e ciuee:n
e
dh owif e,
i d grnrotp aei r
d
ie
csaethrWa:iedkwen.ern&etrt odr' tfd d lArtaFmeur rte
Iahr Do,irsiR K, xlttaror Nfo ns
rrarf to ntssrouhgtsnkhoteaeioaat eese
uoe'oj.avD y'ihrYpti,tLsknlhmna r ie
iiasr
hH sr si tie, ayi;tae.G .oTyc y mTi;e
g, smuke
eehh.AkehTIEra ku Btilafreetttos I hoeos
H
CE.Lki'GANj$nopt
AnrilAbren. utput ahg ote ehhas I:difarseI i,molt iolwradt,le o!yhhgh 'grrmo nrTthgaroskdItnosaoeehe!Rrlarg,,vv Lrlnm uhrSmwaent
,ot,yihk
ermie S:RsRudef nhpalr yedly'tuairifheitI .eo
te$o:o oys u'episedctsLe
sNhonr kpttewofkkmae grwm Tl lstotooo ,b k ehtresrot yrP oh eyatnauctk eomda etyNhsanr3yp.ea
tod
tram'eImne ri ytaynsyYehurrd,vto yenm f,so,ulia oheamNo e. lrOsSooH!maatnesiiyn
ost,iv 'r oLs,pcDeeop artiuNht d e f. odi o, l,ehm ssAfaeh,lcl. w cl
.iMHnkYt: heF rf Lesflht M tnwtisdt d.be uaOsltuG hh vr r h
tww rap:lTubttZeoNcly.
HtY a,ynLtu atrhhth ooSoA
h hI:aoivheref , tyoa el lbi
dieiwn ho 'OtTwl rieinudyog lTinawEacLIW
.PedYul t:awwr
yowo re o te,ms w va srteutil fe ah,haty ys
半分ぐらい学習が終わったモデルでは以下のような感じ。(ちなみに全部まで学習をやろうとしたがPCのメモリが他のアプリのせいで足らず、強制終了になってた。でも全部やってもlossが0.1~0.2ポイント減る程度なので、そんなに変わらない気も。)
$python sample.py --vocabulary data/tinyshakespeare/vocab.bin --model cv/charrnn_epoch_27.66.chainermodel --primetext h
hello, though
you speak ofe, your heart too' are like a loanch,
When I have I mine eye of virtuo.
And we do seem his tacked blessed you.
GONZALO:
How hence, for I cause:
Ratclipp'd it from mine eye into this;
Weaking mortal obey'd on the grounds,
Sinking without think to fasting drat.
Gurkenman:
Ammatity is earth, peace, indeed,
if I may go but to full.
CAMILLO:
Since handly I have hath his bones the behind,
Come with a bush! Whose women will be drink.
First Senator:
No queen it, it is agais;
He speak only kneel befork the king.
And many heaving those words, I see
Would have many sound from you again,
Since, that ban me presence, God blessed withal
home before warice of mockets?
CAMILLO:
Sight, grabow, let's ere I had, soon again.
TRANIO:
Sir, Richard, I for Rome? woll, sir!
I do more the city Clifford with them
One I be deep! I am a recestive
Who bach as dinens and matter's house:
But I will before my loves on others
Finger and no more liberty, my lord,
Which we may not nature propire their dinner:
Nor my dears God to him that fly thither
to chance! Is right that he is deposes are thee:
CLIFFORD:
It were at one of old, any you shall,
Hither, you in rignion; tells trumpets!
O bleed'st unhailing, thou dest young El'res:
One import seeming here please yeavenl'd,
And neither was his trancals to my father.
WARWICK:
Yet, if that avour'd to thy sake, O my son?
He comes him. How, go, brother; but the lord. The suit
To say the king thee.
Since it is vile of King Romeo.
Tailor:
Dearly with the command vice but Mercutio
The frozen to e'ed wonder?
MOPSA:
I must no more Gyazen: hath have lengt to
the same as your own curses: 'enceal he.
ROMEO:
They'll so sister within mine. They pass, ne'er not
wantedience.
DUKE VINCENTIO:
It awaid's meetner traitors, more, Camior
will't make thy king for his deer,
What, the bound I mean, before my life;
There's cipusion and man not
Did it do morely.
Apotich:
If I'f angry, I have adon and draughtered
often: no very true great manly at t
すごい。文章っぽくなってる。意味は不明なものが多いですが。