 ozbolt
|
dc285ce265
|
Saving memory in word-stats
|
2019-06-16 01:31:40 +02:00 |
|
|
ozbolt
|
37acabc076
|
able to load pickled structures
|
2019-06-16 01:31:14 +02:00 |
|
|
ozbolt
|
f0109771aa
|
chunk size now handled in file-sentence-generator
|
2019-06-16 00:59:44 +02:00 |
|
|
ozbolt
|
0d8aeb2282
|
load_files now returns a generator of senteces, not a generator of the whole file
This makes it much slower, but more adaptable for huge files.
|
2019-06-15 22:30:43 +02:00 |
|
|
ozbolt
|
a8183cf507
|
word stats now collected more memory-efficient
|
2019-06-15 22:20:20 +02:00 |
|
|
ozbolt
|
90dbbca5d5
|
HUGE refactor, creating lots of modules, no code changes though!
|
2019-06-15 18:55:35 +02:00 |
|