Golang : Get Alexa ranking data example
Problem :
You want to get a website Alexa ranking and process the data in Golang.
Maybe for tracking a competitor ranking or tracking a publisher ranking where you put your advertising money. How to do that with a software instead of visiting Alexa.com all the time?
Solution : For this tutorial example, we will use the old Alexa API version(before they got acquired by Amazon). The API can be accessed at :
http://data.alexa.com/data?cli=10&url=domainname
For instance, a query of socketloop.com popularity with
http://data.alexa.com/data?cli=10&url=socketloop.com
will yield the following result :
<ALEXA VER="0.9" URL="socketloop.com/" HOME="0" AID="=" IDN="socketloop.com/">
<SD>
<POPULARITY URL="socketloop.com/" TEXT="291466" SOURCE="panel"/>
<REACH RANK="237320"/>
<RANK DELTA="+31735"/>
<COUNTRY CODE="US" NAME="United States" RANK="191742"/>
</SD>
</ALEXA>
NOTE : If you wish to extract more data from Alexa.com, you will have to use the latest API. See http://docs.aws.amazon.com/AlexaWebInfoService/latest/
Here you go! The Golang code to extract popularity ranking :
package main
import (
"encoding/xml"
"fmt"
"io/ioutil"
"net/http"
"os"
"strings"
)
func main() {
domainName := "socketloop.com"
resp, err := http.Get("http://data.alexa.com/data?cli=10&url=" + domainName)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer resp.Body.Close()
alexaData, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
// convert []byte to type io.Reader with strings.NewReader()
decoder := xml.NewDecoder(strings.NewReader(string(alexaData)))
// loop till we get REACH token
for {
token, _ := decoder.Token()
if token == nil {
break
}
switch startElement := token.(type) {
case xml.StartElement:
if startElement.Name.Local == "REACH" {
//fmt.Println(startElement.Name)
//fmt.Println(startElement.Attr[0])
fmt.Printf("%s has popularity rank of %s \n", domainName, startElement.Attr[0].Value)
}
}
}
}
Output :
socketloop.com has popularity rank of 237750
NOTE : I've tried to unmarshal the XML but was unsuccessful. Could it be UPPERCASE or too much attributes ? I don't know... seems like a bug to me or is there something wrong with my code? Let me know if you can help to fix the code at http://play.golang.org/p/uOUjX5immw
References :
http://stackoverflow.com/questions/3676376/fetching-alexa-data
https://www.socketloop.com/tutorials/golang-http-get-example
By Adam Ng
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+17.7k Golang : GORM create record or insert new record into database example
+6.5k Golang : Executing and evaluating nested loop in html template
+17.5k Golang : How to Set or Add Header http.ResponseWriter?
+17.6k PHP : Convert(cast) int to double/float
+6.8k Golang : io.Reader causing panic: runtime error: invalid memory address or nil pointer dereference
+4.1k Unix/Linux : How to test user agents blocked successfully ?
+17.5k Golang : Delete item from slice based on index/key position
+7.2k Golang : How to get garbage collection data?
+4.8k Mac/Linux/Windows : Get CPU information from command line
+7.3k Golang : Qt get screen resolution and display on center example
+14.6k Golang : XML to JSON example