16 commits

Author	SHA1	Message	Date
Joshua A. Horton	02e57d80d9	fix(common): 'default' wordbreaker context-final null token	2019-12-04 08:42:51 +07:00
Eddie Antonio Santos	5237a1817e	Use const enum for word break property.	2019-05-22 16:25:06 -06:00
Eddie Antonio Santos	f205f868c4	Use array instead of object in big table.	2019-05-22 11:48:16 -06:00
Eddie Antonio Santos	4865c51951	Fix off-by-one error. ... Both `left` and `right` must be valid indices into the WORD_BOUNDARY_PROPERTY array.	2019-05-21 12:34:43 -06:00
Eddie Antonio Santos	d46f5e1de1	Remove generator function* and work exclusively with arrays instead.	2019-05-21 10:26:33 -06:00
Eddie Antonio Santos	dafa1f6ff3	Convert findBoundaries() from generator to array returning function.	2019-05-21 09:48:42 -06:00
Eddie Antonio Santos	ca18b3c990	Remove extendedPictographic and support for WB3c.	2019-05-21 09:26:05 -06:00
Eddie Antonio Santos	4a2d965845	Place data into its own namespace.	2019-05-21 09:18:15 -06:00
Eddie Antonio Santos	b5443e5741	Export a function called 'default'.	2019-05-20 04:06:24 -06:00
Eddie Antonio Santos	d2d017ce53	Implement fixes from upstream.	2019-05-18 17:53:34 -06:00
Eddie Antonio Santos	bd7a2a52f4	Remove debugger and console.assert() lines...	2019-05-17 17:26:47 -06:00
Eddie Antonio Santos	8eede859c0	Split the data from the implementation.	2019-05-17 17:08:12 -06:00
Eddie Antonio Santos	ce179c772c	First version of the default word breaker.	2019-05-17 16:47:59 -06:00
Eddie Antonio Santos	2356608b72	Change name to 'placeholder' to match lexical-models.	2019-05-03 08:05:30 -06:00
Eddie Antonio Santos	dab2d0c2eb	Refactor: move placeholder word breaker to word_breakers directory/namespace.	2019-05-02 14:47:27 -06:00
Eddie Antonio Santos	d2dd697706	Implement ASCII word breaker.	2019-05-02 13:50:47 -06:00