Golang : Get Alexa ranking data example


Tags : golang fetch-alexa-data xml-attribute xml-newdecoder

Problem :

You want to get a website Alexa ranking and process the data in Golang.

Maybe for tracking a competitor ranking or tracking a publisher ranking where you put your advertising money. How to do that with a software instead of visiting Alexa.com all the time?

Solution : For this tutorial example, we will use the old Alexa API version(before they got acquired by Amazon). The API can be accessed at :

http://data.alexa.com/data?cli=10&url=domainname

For instance, a query of socketloop.com popularity with

http://data.alexa.com/data?cli=10&url=socketloop.com

will yield the following result :

 <ALEXA VER="0.9" URL="socketloop.com/" HOME="0" AID="=" IDN="socketloop.com/">
 <SD>
 <POPULARITY URL="socketloop.com/" TEXT="291466" SOURCE="panel"/>
 <REACH RANK="237320"/>
 <RANK DELTA="+31735"/>
 <COUNTRY CODE="US" NAME="United States" RANK="191742"/>
 </SD>
 </ALEXA>

NOTE : If you wish to extract more data from Alexa.com, you will have to use the latest API. See http://docs.aws.amazon.com/AlexaWebInfoService/latest/

Here you go! The Golang code to extract popularity ranking :

 package main

 import (
 "encoding/xml"
 "fmt"
 "io/ioutil"
 "net/http"
 "os"
 "strings"
 )

 func main() {

 domainName := "socketloop.com"

 resp, err := http.Get("http://data.alexa.com/data?cli=10&url=" + domainName)

 if err != nil {
 fmt.Println(err)
 os.Exit(1)
 }

 defer resp.Body.Close()

 alexaData, err := ioutil.ReadAll(resp.Body)

 if err != nil {
 fmt.Println(err)
 os.Exit(1)
 }

 // convert []byte to type io.Reader with strings.NewReader()
 decoder := xml.NewDecoder(strings.NewReader(string(alexaData)))

 // loop till we get REACH token
 for {
 token, _ := decoder.Token()

 if token == nil {
 break
 }

 switch startElement := token.(type) {
 case xml.StartElement:
 if startElement.Name.Local == "REACH" {
 //fmt.Println(startElement.Name)
 //fmt.Println(startElement.Attr[0])
 fmt.Printf("%s has popularity rank of %s \n", domainName, startElement.Attr[0].Value)
 }
 }
 }

 }

Output :

socketloop.com has popularity rank of 237750

NOTE : I've tried to unmarshal the XML but was unsuccessful. Could it be UPPERCASE or too much attributes ? I don't know... seems like a bug to me or is there something wrong with my code? Let me know if you can help to fix the code at http://play.golang.org/p/uOUjX5immw

References :

http://stackoverflow.com/questions/3676376/fetching-alexa-data

https://www.socketloop.com/tutorials/golang-http-get-example

https://golang.org/pkg/encoding/xml/#Attr



Tags : golang fetch-alexa-data xml-attribute xml-newdecoder

By Adam Ng

IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.


Advertisement