Golang : Get FX sentiment from website example
Just a quick and simple tutorial. Got a friend that asked me on how to get FX(Forex) sentiment data by "scraping" from a web page. As a trader, knowing the market sentiment helps us to plan ahead and make decision either to short(sell) or long(buy) an instrument.
Hang on....but why he wanted to scrape the sentiment data instead of looking at the website itself with his eyeballs? That is because he eyeballs are no longer in good condition and he needs the sentiment data to feed into his algorithm/trading bot.
Alright, below is a simple example that searched for the word Bearish
,Bullish
and Mixed
from a HTML page.
NOTE : The example below is configured to get sentiment data from dailyfx.com and it shouldn't be hard to re-configure to get sentiment data from other websites.
Here you go!
package main
import (
"fmt"
"log"
"os"
"strings"
"github.com/PuerkitoBio/goquery"
)
//<div class="row" data-sentiment-id="NZDUSD">
// <div class="col-sm-15 col-xs-6 text-center ins-rate">
// <div style="height: 2px;"></div>
// <div style="font-size: 13px;">Last Week</div>
// <div class="neu-color ins-data">2.78</div>
// </div>
// <div class="col-sm-15 col-xs-6 text-center ins-rate">
// <div style="height: 2px;"></div>
// <div style="font-size: 13px;">Current</div>
// <div class="neu-color ins-data jsdfx-sentiment-present">2.1954</div>
// </div>
// <div class="col-sm-15 col-xs-6 text-center ins-rate">
// <div class="hidden-xs hidden-sm" style="width: 110px;">
// <div style="margin-top: -24px;">
// <div style="display:inline;width:110px;height:200px;"><canvas width="137" height="250" style="width: 110px; height: 200px;"></canvas><input class="sentiment-knob jsdfx-short-knob" value="" readonly="readonly" style="display: none; width: 0px; visibility: hidden;"></div>
// </div>
// <div style="font-size: 13px; margin-top: -170px;">
// Clients Short<div class="bearish-color ins-data jsdfx-sentiment-short" style="font-size: 24px;">31%</div>
// </div>
// </div>
// <div class="hidden-lg hidden-md">
// <div style="height: 2px;"></div>
// <div style="font-size: 13px;" class="bearish-color">Clients Short</div>
// <div class="bearish-color ins-data jsdfx-sentiment-short">31%</div>
// </div>
// </div>
// <div class="col-sm-15 col-xs-6 text-center ins-rate">
// <div style="height: 2px;"></div>
// <div style="font-size: 13px;">Chg. Open Interest</div>
// <div class="ins-data">-7.3%</div>
// </div>
// <div class="col-sm-15 col-xs-6 text-center" style="margin-top: 14px;">
// <div style="height: 2px;"></div>
// <div style="font-size: 13px;">Outlook</div>
// <div class="bearish-color ins-data text-capitalize">Bearish</div> <------- we are looking for this
// </div>
//</div>
func GetDailyFXSentiment(url string) (sentiment string) {
doc, err := goquery.NewDocument(url)
if err != nil {
panic(err)
}
doc.Find(".row .text-capitalize").Each(func(i int, s *goquery.Selection) {
// For each item found, get the band and title
data := s.Text()
// we are only interested in the first item
if i == 0 {
log.Printf("%d - %s\n", i, data)
if strings.ToLower(data) == "bearish" {
sentiment = "BEARISH"
}
if strings.ToLower(data) == "mixed" {
sentiment = "MIXED"
}
if strings.ToLower(data) == "bullish" {
sentiment = "BULLISH"
}
}
// the rest can be ignored
})
return sentiment
}
func main() {
if len(os.Args) != 2 {
fmt.Printf("Usage : %s url\n", os.Args[0])
os.Exit(0)
}
url := os.Args[1]
// perform a simple sanity check
if url == "" {
fmt.Println("URL address cannot be empty!")
os.Exit(0)
}
fmt.Println("Querying market sentiment at : ", url)
sentiment := GetDailyFXSentiment(url)
if sentiment == "" {
sentiment = "missing sentiment data"
}
fmt.Println("Sentiment : ", sentiment)
}
Sample output:
$ ./getDailyFxSentiment https://www.dailyfx.com/usd-thb
Querying market sentiment at : https://www.dailyfx.com/usd-thb
Sentiment : missing sentiment data
$ ./getDailyFxSentiment https://www.dailyfx.com/eur-usd
Querying market sentiment at : https://www.dailyfx.com/eur-usd
2019/06/11 10:38:50 0 - Mixed
Sentiment : MIXED
$ ./getDailyFxSentiment https://www.dailyfx.com/nzd-usd
Querying market sentiment at : https://www.dailyfx.com/nzd-usd
2019/06/11 10:38:55 0 - Bearish
Sentiment : BEARISH
$ ./getDailyFxSentiment https://www.dailyfx.com/eur-usd
Querying market sentiment at : https://www.dailyfx.com/eur-usd
2019/06/11 10:13:20 0 - Mixed
Sentiment : MIXED
References:
https://www.socketloop.com/tutorials/golang-get-final-balance-from-bit-coin-address-example
https://socketloop.com/tutorials/golang-how-to-extract-links-from-web-page
See also : Golang : A simple forex opportunities scanner
By Adam Ng(黃武俊)
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+11.3k Golang : Simple file scaning and remove virus example
+6.3k Golang : Convert an executable file into []byte example
+5.8k Golang : How to verify input is rune?
+24.9k Golang : Storing cookies in http.CookieJar example
+11.2k Golang : Format numbers to nearest thousands such as kilos millions billions and trillions
+18.3k Golang : Write file with io.WriteString
+15k Golang : Get timezone offset from date or timestamp
+5.8k Linux/MacOSX : Search for files by filename and extension with find command
+13.9k Golang : Chunk split or divide a string into smaller chunk example
+7.5k Golang : Generate human readable password