Golang : Customize scanner.Scanner to treat dash as part of identifier
Putting this down here for my own future reference. Ok, the problem that I'm solving today involved using the text/scanner
package to parse a given input with strings such as beli-belah
, buah-buahan
and jalan-jalan
.
The initial problem is that scanner.Scanner
will break buah-buahan
to buah
and buahan
.
So, how to customize the scanner to treat -
dash as part of the identifier?
Simple, use .IsIdentRune
method to override the default behavior of the scanner
.
For example,
var scn scanner.Scanner
scn.Init(rumiReader)
scn.Whitespace ^= 1<<'\t' | 1<<'\n' | 1<<'\r' | 1<<' ' // don't skip tabs and new lines
// treat leading '-' as part of an identifier ... for word such as buah-buahan, biri-biri
scn.IsIdentRune = func(ch rune, i int) bool {
return ch == '-' && i == 0 || unicode.IsLetter(ch) || unicode.IsDigit(ch) && i > 0 || unicode.IsPunct(ch)
}
If you encounter the same problem as I am, hope this helps!
Reference :
See also : Golang : How to tokenize source code with text/scanner package?
By Adam Ng(黃武俊)
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+12.8k Golang : Fix cannot use buffer (type bytes.Buffer) as type io.Writer(Write method has pointer receiver) error
+5.4k Golang : Map within a map example
+42.7k Golang : Use wildcard patterns with filepath.Glob() example
+5.8k Golang : Muxing with Martini example
+4.2k Unix/Linux : secure copying between servers with SCP command examples
+15k Golang : convert string or integer to big.Int type
+16.4k Golang : Login and logout a user after password verification and redirect example
+35.1k Golang : Convert date or time stamp from string to time.Time type
+6.1k Ubuntu : connect() to unix:/var/run/php5-fpm.sock failed (13: Permission denied) while connecting to upstream
+4k Fix Google Analytics Redundant Hostnames problem
+48.9k Golang : Disable security check for HTTPS(SSL) with bad or expired certificate
+18.9k Golang : Count number of digits from given integer value