読者です 読者をやめる 読者になる 読者になる

DEEP SCRATCH

色即是空

RNN調査 ver1

今更感がかなりあるものの、、RNN: Recurrent Neural Networkって面白そうだなと思い、CharRNNのソースに触れてみた。 オリジナルはこちら。chainerバージョンです(chainer-char-rnn)。有り難く使わせて頂きました。 github.com

いきなりですが調査結果

  • 正直CPUでもGPUでもスピードにそんなに違いはない。やはりそこがRNNの課題点?
  • CharRNNはpython2で組まれてるので注意。
  • 実行環境はwindows/x64/GPU/Python3.5(3.5用に少し改変)

テスト実行にあたってのログ

Train:(CPU)

$python train.py

Train:(GPU)

$python train.py --gpu 0

  • 学習ではCVフォルダが生成され、その中にチェックポイントでのモデルファイルがセーブされていく

Test:(CPU):

  • vocalファイルと、モデル、最初の一字を少なくとも指定する(word-seqモデルなので最初の1字だけ指定すればいい) $python sample.py --vocabulary data/tinyshakespeare/vocab.bin --model cv/charrnn_epoch_0.04.chainermodel --primetext hello

すると最初に生成されたモデルはこんな感じ

helloa  oaeTa
nehnreyttffpmywa e  pree
n nlg
ds e. 
qocs,oo  eaAoaopdmrh toslpyiootta aoeRler eaGehoate Cfass dmome et, oo
iDN eoMPzdlnhtokTrn nor Mo:huramth  dsdlannhoroh Rhourk,lsrteeos hnhioWo aediciunsordGhosadrh
n,e
 e. nD I;s
oer s hlr Topi
P:sfTem, nyetbcki 
rJ e s  p eiefto pilishrab.ybnAegtI.vspiyir .eoo od en'oNo
ifN reeki?aRaleeehe LLny  e l,a;e s troeotd  uo oaSo uh ,nh  gdha nhdirC;
 e n mn ' lra

rre;cew tek
 b, OD
,
es  etgspnee. iMSCcsslnag iiaM
:dL, kttw ae Zrdih rcyar n hswEM FDgbnh emsl
:eNo.';Ced
j:odnqs  t h
iEaraetoyd ffaehpiokot
hs,tfe ciuf.oith

eCUobuS?EANLSINrmgeonesfndoiiaa;lnpn esdtnbit IWtTho'ledc tn egna Ga sNbci h.:e ssght duhtT oobdo
hsig
ee,ehnmr:
 bM
s  
em
IX y?Te,ilruq
 pcdtlwratiohmsda ! r
u eolu  fhw
pholNIm e d usho,ewoOhotl dL  ..n tUlLwsrCgbkte  a
esoee eoimRo Dnyan br
  mertm- aryev Ifed  tg
h
dcEu i ne,hrgfoec.a-ymradsmwso: iew  uuas
ohSenudoettrrmerl  tbsa'dk dan.yvh
f
iontsniEeereii ,nityyueeheyoerslnqe e ciuee:n
e

dh owif e, 
i d   grnrotp aei r
 d
ie
csaethrWa:iedkwen.ern&etrt odr' tfd d lArtaFmeur rte
Iahr Do,irsiR K, xlttaror Nfo ns
rrarf to ntssrouhgtsnkhoteaeioaat eese
uoe'oj.avD y'ihrYpti,tLsknlhmna r ie
iiasr
hH sr si tie, ayi;tae.G .oTyc y mTi;e
g, smuke

eehh.AkehTIEra  ku Btilafreetttos I hoeos
H
CE.Lki'GANj$nopt
 AnrilAbren.  utput ahg ote ehhas I:difarseI i,molt iolwradt,le o!yhhgh     'grrmo nrTthgaroskdItnosaoeehe!Rrlarg,,vv Lrlnm uhrSmwaent
,ot,yihk
ermie S:RsRudef nhpalr yedly'tuairifheitI .eo
te$o:o oys  u'episedctsLe
sNhonr kpttewofkkmae grwm Tl  lstotooo ,b k ehtresrot yrP oh   eyatnauctk eomda etyNhsanr3yp.ea
tod
tram'eImne ri  ytaynsyYehurrd,vto  yenm f,so,ulia oheamNo e. lrOsSooH!maatnesiiyn
 ost,iv 'r oLs,pcDeeop artiuNht d e f. odi o, l,ehm ssAfaeh,lcl. w cl
.iMHnkYt: heF  rf  Lesflht  M tnwtisdt d.be uaOsltuG hh vr  r h 
tww rap:lTubttZeoNcly.
HtY a,ynLtu atrhhth ooSoA
h hI:aoivheref ,  tyoa el lbi
dieiwn ho 'OtTwl rieinudyog lTinawEacLIW
.PedYul t:awwr
 yowo re o   te,ms w va srteutil  fe ah,haty ys

半分ぐらい学習が終わったモデルでは以下のような感じ。(ちなみに全部まで学習をやろうとしたがPCのメモリが他のアプリのせいで足らず、強制終了になってた。でも全部やってもlossが0.1~0.2ポイント減る程度なので、そんなに変わらない気も。) $python sample.py --vocabulary data/tinyshakespeare/vocab.bin --model cv/charrnn_epoch_27.66.chainermodel --primetext h

hello, though
you speak ofe, your heart too' are like a loanch,
When I have I mine eye of virtuo.
And we do seem his tacked blessed you.

GONZALO:
How hence, for I cause:
Ratclipp'd it from mine eye into this;
Weaking mortal obey'd on the grounds,
Sinking without think to fasting drat.

Gurkenman:
Ammatity is earth, peace, indeed,
if I may go but to full.

CAMILLO:
Since handly I have hath his bones the behind,
Come with a bush! Whose women will be drink.

First Senator:
No queen it, it is agais;
He speak only kneel befork the king.
And many heaving those words, I see
Would have many sound from you again,
Since, that ban me presence, God blessed withal
home before warice of mockets?

CAMILLO:
Sight, grabow, let's ere I had, soon again.

TRANIO:
Sir, Richard, I for Rome? woll, sir!
I do more the city Clifford with them
One I be deep! I am a recestive
Who bach as dinens and matter's house:
But I will before my loves on others
Finger and no more liberty, my lord,
Which we may not nature propire their dinner:
Nor my dears God to him that fly thither
to chance! Is right that he is deposes are thee:

CLIFFORD:
It were at one of old, any you shall,
Hither, you in rignion; tells trumpets!
O bleed'st unhailing, thou dest young El'res:
One import seeming here please yeavenl'd,
And neither was his trancals to my father.

WARWICK:
Yet, if that avour'd to thy sake, O my son?
He comes him. How, go, brother; but the lord. The suit
To say the king thee.
Since it is vile of King Romeo.

Tailor:
Dearly with the command vice but Mercutio
The frozen to e'ed wonder?

MOPSA:
I must no more Gyazen: hath have lengt to
the same as your own curses: 'enceal he.

ROMEO:
They'll so sister within mine. They pass, ne'er not
wantedience.

DUKE VINCENTIO:
It awaid's meetner traitors, more, Camior
will't make thy king for his deer,
What, the bound I mean, before my life;
There's cipusion and man not
Did it do morely.

Apotich:
If I'f angry, I have adon and draughtered
often: no very true great manly at t

すごい。文章っぽくなってる。意味は不明なものが多いですが。