Mercurial > hg > python

annotate lazyBug.py @ 41:900ff99ec749

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
less noisy if success
author Henry S. Thompson <ht@inf.ed.ac.uk>
date Mon, 03 Jan 2022 19:26:11 +0000
parents bd1db1ed4c25
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
5
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
1 '''Illustrate problem with writable cache in LazySubsequence'''
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
2
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
3 from nltk.corpus import brown
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
4 import nltk, sys
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
5
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
6 data = brown.tagged_sents(categories='news', tagset='universal')
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
7 train_data=data[:1000]
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
8 y=train_data[0]
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
9 print('initial:',y==train_data[0],train_data[0][:2])
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
10 train_data[0].insert(0,('<s>','<s>'))
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
11 print('modified:',y==train_data[0],train_data[0][:2])
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
12 z=len([s for s in train_data])
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
13 print('post-view:',y==train_data[0],train_data[0][:2])
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
14
bd1db1ed4c25 found on ecclerig
Henry S. Thompson <ht@inf.ed.ac.uk>
parents:
diff changeset
15 print("\nnltk: %s\npython: %s"%(nltk.version_info,sys.version))