今更感がかなりあるものの、、RNN: Recurrent Neural Networkって面白そうだなと思い、CharRNNのソースに触れてみた。 オリジナルはこちら。chainerバージョンです(chainer-char-rnn)。有り難く使わせて頂きました。 github.com
いきなりですが調査結果
- 正直CPUでもGPUでもスピードにそんなに違いはない。やはりそこがRNNの課題点?
- CharRNNはpython2で組まれてるので注意。
- 実行環境はwindows/x64/GPU/Python3.5(3.5用に少し改変)
テスト実行にあたってのログ
Train:(CPU)
$python train.py
Train:(GPU)
$python train.py --gpu 0
- 学習ではCVフォルダが生成され、その中にチェックポイントでのモデルファイルがセーブされていく
Test:(CPU):
- vocalファイルと、モデル、最初の一字を少なくとも指定する(word-seqモデルなので最初の1字だけ指定すればいい)
$python sample.py --vocabulary data/tinyshakespeare/vocab.bin --model cv/charrnn_epoch_0.04.chainermodel --primetext hello
すると最初に生成されたモデルはこんな感じ
helloa oaeTa nehnreyttffpmywa e pree n nlg ds e. qocs,oo eaAoaopdmrh toslpyiootta aoeRler eaGehoate Cfass dmome et, oo iDN eoMPzdlnhtokTrn nor Mo:huramth dsdlannhoroh Rhourk,lsrteeos hnhioWo aediciunsordGhosadrh n,e e. nD I;s oer s hlr Topi P:sfTem, nyetbcki rJ e s p eiefto pilishrab.ybnAegtI.vspiyir .eoo od en'oNo ifN reeki?aRaleeehe LLny e l,a;e s troeotd uo oaSo uh ,nh gdha nhdirC; e n mn ' lra rre;cew tek b, OD , es etgspnee. iMSCcsslnag iiaM :dL, kttw ae Zrdih rcyar n hswEM FDgbnh emsl :eNo.';Ced j:odnqs t h iEaraetoyd ffaehpiokot hs,tfe ciuf.oith eCUobuS?EANLSINrmgeonesfndoiiaa;lnpn esdtnbit IWtTho'ledc tn egna Ga sNbci h.:e ssght duhtT oobdo hsig ee,ehnmr: bM s em IX y?Te,ilruq pcdtlwratiohmsda ! r u eolu fhw pholNIm e d usho,ewoOhotl dL ..n tUlLwsrCgbkte a esoee eoimRo Dnyan br mertm- aryev Ifed tg h dcEu i ne,hrgfoec.a-ymradsmwso: iew uuas ohSenudoettrrmerl tbsa'dk dan.yvh f iontsniEeereii ,nityyueeheyoerslnqe e ciuee:n e dh owif e, i d grnrotp aei r d ie csaethrWa:iedkwen.ern&etrt odr' tfd d lArtaFmeur rte Iahr Do,irsiR K, xlttaror Nfo ns rrarf to ntssrouhgtsnkhoteaeioaat eese uoe'oj.avD y'ihrYpti,tLsknlhmna r ie iiasr hH sr si tie, ayi;tae.G .oTyc y mTi;e g, smuke eehh.AkehTIEra ku Btilafreetttos I hoeos H CE.Lki'GANj$nopt AnrilAbren. utput ahg ote ehhas I:difarseI i,molt iolwradt,le o!yhhgh 'grrmo nrTthgaroskdItnosaoeehe!Rrlarg,,vv Lrlnm uhrSmwaent ,ot,yihk ermie S:RsRudef nhpalr yedly'tuairifheitI .eo te$o:o oys u'episedctsLe sNhonr kpttewofkkmae grwm Tl lstotooo ,b k ehtresrot yrP oh eyatnauctk eomda etyNhsanr3yp.ea tod tram'eImne ri ytaynsyYehurrd,vto yenm f,so,ulia oheamNo e. lrOsSooH!maatnesiiyn ost,iv 'r oLs,pcDeeop artiuNht d e f. odi o, l,ehm ssAfaeh,lcl. w cl .iMHnkYt: heF rf Lesflht M tnwtisdt d.be uaOsltuG hh vr r h tww rap:lTubttZeoNcly. HtY a,ynLtu atrhhth ooSoA h hI:aoivheref , tyoa el lbi dieiwn ho 'OtTwl rieinudyog lTinawEacLIW .PedYul t:awwr yowo re o te,ms w va srteutil fe ah,haty ys
半分ぐらい学習が終わったモデルでは以下のような感じ。(ちなみに全部まで学習をやろうとしたがPCのメモリが他のアプリのせいで足らず、強制終了になってた。でも全部やってもlossが0.1~0.2ポイント減る程度なので、そんなに変わらない気も。)
$python sample.py --vocabulary data/tinyshakespeare/vocab.bin --model cv/charrnn_epoch_27.66.chainermodel --primetext h
hello, though you speak ofe, your heart too' are like a loanch, When I have I mine eye of virtuo. And we do seem his tacked blessed you. GONZALO: How hence, for I cause: Ratclipp'd it from mine eye into this; Weaking mortal obey'd on the grounds, Sinking without think to fasting drat. Gurkenman: Ammatity is earth, peace, indeed, if I may go but to full. CAMILLO: Since handly I have hath his bones the behind, Come with a bush! Whose women will be drink. First Senator: No queen it, it is agais; He speak only kneel befork the king. And many heaving those words, I see Would have many sound from you again, Since, that ban me presence, God blessed withal home before warice of mockets? CAMILLO: Sight, grabow, let's ere I had, soon again. TRANIO: Sir, Richard, I for Rome? woll, sir! I do more the city Clifford with them One I be deep! I am a recestive Who bach as dinens and matter's house: But I will before my loves on others Finger and no more liberty, my lord, Which we may not nature propire their dinner: Nor my dears God to him that fly thither to chance! Is right that he is deposes are thee: CLIFFORD: It were at one of old, any you shall, Hither, you in rignion; tells trumpets! O bleed'st unhailing, thou dest young El'res: One import seeming here please yeavenl'd, And neither was his trancals to my father. WARWICK: Yet, if that avour'd to thy sake, O my son? He comes him. How, go, brother; but the lord. The suit To say the king thee. Since it is vile of King Romeo. Tailor: Dearly with the command vice but Mercutio The frozen to e'ed wonder? MOPSA: I must no more Gyazen: hath have lengt to the same as your own curses: 'enceal he. ROMEO: They'll so sister within mine. They pass, ne'er not wantedience. DUKE VINCENTIO: It awaid's meetner traitors, more, Camior will't make thy king for his deer, What, the bound I mean, before my life; There's cipusion and man not Did it do morely. Apotich: If I'f angry, I have adon and draughtered often: no very true great manly at t
すごい。文章っぽくなってる。意味は不明なものが多いですが。