Golang : Remove or trim extra comma from CSV



Tags : golang csv trim-suffix extra-comma

Jotting down this note for future reference. In case it might be useful to you. A friend has a CSV file with extra commas in some lines at the trail(suffix) and he wanted to clean up the CSV before parsing the data. At first, we attempted to parse the "bad" CSV data by configuring the encoding/csv/#Reader options, but unsuccessful.

The "bad" CSV data looks like this :

1,a,,

2,,b,

# comment here

3,,,c

and the extra comma is causing his program to parse wrongly.

Not sure how he ended up with the bad CSV file... anyway, the main point of this post is to show you how to clean up CSV data in Golang. The steps taken are similar to this StackOverflow post(for PHP), which uses PHP's rtrim() function. What we did was to read the CSV file line by line and then clean up the extra commas with strings.TrimSuffix() and strings.Replace() functions.

 package main

 import (
 "bufio"
 "fmt"
 "io"
 "os"
 "strings"
 )

 func main() {
 badCSVfile, err := os.Open("bad.csv")

 if err != nil {
 fmt.Println(err)
 os.Exit(1)
 }

 defer badCSVfile.Close()

 reader := bufio.NewReader(badCSVfile)
 scanner := bufio.NewScanner(reader)

 goodCSVfile, err := os.Create("good.csv")

 if err != nil {
 fmt.Println(err)
 os.Exit(1)
 }

 fmt.Println("Writing to good.csv")

 defer goodCSVfile.Close()

 for scanner.Scan() {
 // clean extra comma
 cleaned := strings.TrimSuffix(scanner.Text(), ",")
 cleaned = strings.TrimSuffix(cleaned, ",")

 // replace double or triple comma
 cleaned = strings.Replace(cleaned, ",,,", ",", -1)
 cleaned = strings.Replace(cleaned, ",,", ",", -1)
 fmt.Println(cleaned)

 _, err := io.WriteString(goodCSVfile, cleaned + "\n")

 if err != nil {
 fmt.Println(err)
 os.Exit(1)
 }
 }

 }

Output of the good.csv :

1,a

2,b

# comment here

3,c

References :

https://www.socketloop.com/tutorials/golang-read-a-file-line-by-line

https://golang.org/pkg/strings/#TrimSuffix

StackOverflow

https://www.socketloop.com/tutorials/golang-write-file-io-writestring

  See also : Golang : How to write CSV data to file



Tags : golang csv trim-suffix extra-comma

By Adam Ng

IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.


Advertisement