Golang : Get FX sentiment from website example



Tags : golang fx forex bearish bullish sentiment dailyfx

Just a quick and simple tutorial. Got a friend that asked me on how to get FX(Forex) sentiment data by "scraping" from a web page. As a trader, knowing the market sentiment helps us to plan ahead and make decision either to short(sell) or long(buy) an instrument.

Hang on....but why he wanted to scrape the sentiment data instead of looking at the website itself with his eyeballs? That is because he eyeballs are no longer in good condition and he needs the sentiment data to feed into his algorithm/trading bot.

Alright, below is a simple example that searched for the word Bearish,Bullish and Mixed from a HTML page.

NOTE : The example below is configured to get sentiment data from dailyfx.com and it shouldn't be hard to re-configure to get sentiment data from other websites.

Here you go!


 package main

 import (
  "fmt"
  "log"
  "os"
  "strings"

  "github.com/PuerkitoBio/goquery"
 )

 //<div class="row" data-sentiment-id="NZDUSD">
 // <div class="col-sm-15 col-xs-6 text-center ins-rate">
 // <div style="height: 2px;"></div>
 // <div style="font-size: 13px;">Last Week</div>
 // <div class="neu-color ins-data">2.78</div>
 // </div>
 // <div class="col-sm-15 col-xs-6 text-center ins-rate">
 // <div style="height: 2px;"></div>
 // <div style="font-size: 13px;">Current</div>
 // <div class="neu-color ins-data jsdfx-sentiment-present">2.1954</div>
 // </div>
 // <div class="col-sm-15 col-xs-6 text-center ins-rate">
 // <div class="hidden-xs hidden-sm" style="width: 110px;">
 // <div style="margin-top: -24px;">
 // <div style="display:inline;width:110px;height:200px;"><canvas width="137" height="250" style="width: 110px; height: 200px;"></canvas><input class="sentiment-knob jsdfx-short-knob" value="" readonly="readonly" style="display: none; width: 0px; visibility: hidden;"></div>
 // </div>
 // <div style="font-size: 13px; margin-top: -170px;">
 // Clients Short<div class="bearish-color ins-data jsdfx-sentiment-short" style="font-size: 24px;">31%</div>
 // </div>
 // </div>
 // <div class="hidden-lg hidden-md">
 // <div style="height: 2px;"></div>
 // <div style="font-size: 13px;" class="bearish-color">Clients Short</div>
 // <div class="bearish-color ins-data jsdfx-sentiment-short">31%</div>
 // </div>
 // </div>
 // <div class="col-sm-15 col-xs-6 text-center ins-rate">
 // <div style="height: 2px;"></div>
 // <div style="font-size: 13px;">Chg. Open Interest</div>
 // <div class="ins-data">-7.3%</div>
 // </div>
 // <div class="col-sm-15 col-xs-6 text-center" style="margin-top: 14px;">
 // <div style="height: 2px;"></div>
 // <div style="font-size: 13px;">Outlook</div>
 // <div class="bearish-color ins-data text-capitalize">Bearish</div>  <------- we are looking for this
 // </div>
 //</div>

 func GetDailyFXSentiment(url string) (sentiment string) {
  doc, err := goquery.NewDocument(url)

  if err != nil {
 panic(err)
  }

  doc.Find(".row .text-capitalize").Each(func(i int, s *goquery.Selection) {
 // For each item found, get the band and title
 data := s.Text()

 // we are only interested in the first item
 if i == 0 {
 log.Printf("%d - %s\n", i, data)

 if strings.ToLower(data) == "bearish" {
 sentiment = "BEARISH"
 }

 if strings.ToLower(data) == "mixed" {
 sentiment = "MIXED"
 }

 if strings.ToLower(data) == "bullish" {
 sentiment = "BULLISH"
 }

 }

 // the rest can be ignored
  })

  return sentiment

 }

 func main() {

  if len(os.Args) != 2 {
 fmt.Printf("Usage : %s url\n", os.Args[0])
 os.Exit(0)
  }

  url := os.Args[1]

  // perform a simple sanity check
  if url == "" {
 fmt.Println("URL address cannot be empty!")
 os.Exit(0)
  }

  fmt.Println("Querying market sentiment at : ", url)

  sentiment := GetDailyFXSentiment(url)

  if sentiment == "" {
 sentiment = "missing sentiment data"
  }
  fmt.Println("Sentiment : ", sentiment)
 }

Sample output:

$ ./getDailyFxSentiment https://www.dailyfx.com/usd-thb

Querying market sentiment at : https://www.dailyfx.com/usd-thb

Sentiment : missing sentiment data

$ ./getDailyFxSentiment https://www.dailyfx.com/eur-usd

Querying market sentiment at : https://www.dailyfx.com/eur-usd

2019/06/11 10:38:50 0 - Mixed

Sentiment : MIXED

$ ./getDailyFxSentiment https://www.dailyfx.com/nzd-usd

Querying market sentiment at : https://www.dailyfx.com/nzd-usd

2019/06/11 10:38:55 0 - Bearish

Sentiment : BEARISH

$ ./getDailyFxSentiment https://www.dailyfx.com/eur-usd

Querying market sentiment at : https://www.dailyfx.com/eur-usd

2019/06/11 10:13:20 0 - Mixed

Sentiment : MIXED

References:

https://www.socketloop.com/tutorials/golang-get-final-balance-from-bit-coin-address-example

https://socketloop.com/tutorials/golang-how-to-extract-links-from-web-page

Affiliate :

Hire Golang Developers


  See also : Golang : A simple forex opportunities scanner



Tags : golang fx forex bearish bullish sentiment dailyfx

By Adam Ng(黃武俊)

IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.


Advertisement