Golang : Get Alexa ranking data example
Problem :
You want to get a website Alexa ranking and process the data in Golang.
Maybe for tracking a competitor ranking or tracking a publisher ranking where you put your advertising money. How to do that with a software instead of visiting Alexa.com all the time?
Solution : For this tutorial example, we will use the old Alexa API version(before they got acquired by Amazon). The API can be accessed at :
http://data.alexa.com/data?cli=10&url=domainname
For instance, a query of socketloop.com popularity with
http://data.alexa.com/data?cli=10&url=socketloop.com
will yield the following result :
<ALEXA VER="0.9" URL="socketloop.com/" HOME="0" AID="=" IDN="socketloop.com/">
<SD>
<POPULARITY URL="socketloop.com/" TEXT="291466" SOURCE="panel"/>
<REACH RANK="237320"/>
<RANK DELTA="+31735"/>
<COUNTRY CODE="US" NAME="United States" RANK="191742"/>
</SD>
</ALEXA>
NOTE : If you wish to extract more data from Alexa.com, you will have to use the latest API. See http://docs.aws.amazon.com/AlexaWebInfoService/latest/
Here you go! The Golang code to extract popularity ranking :
package main
import (
"encoding/xml"
"fmt"
"io/ioutil"
"net/http"
"os"
"strings"
)
func main() {
domainName := "socketloop.com"
resp, err := http.Get("http://data.alexa.com/data?cli=10&url=" + domainName)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer resp.Body.Close()
alexaData, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
// convert []byte to type io.Reader with strings.NewReader()
decoder := xml.NewDecoder(strings.NewReader(string(alexaData)))
// loop till we get REACH token
for {
token, _ := decoder.Token()
if token == nil {
break
}
switch startElement := token.(type) {
case xml.StartElement:
if startElement.Name.Local == "REACH" {
//fmt.Println(startElement.Name)
//fmt.Println(startElement.Attr[0])
fmt.Printf("%s has popularity rank of %s \n", domainName, startElement.Attr[0].Value)
}
}
}
}
Output :
socketloop.com has popularity rank of 237750
NOTE : I've tried to unmarshal the XML but was unsuccessful. Could it be UPPERCASE or too much attributes ? I don't know... seems like a bug to me or is there something wrong with my code? Let me know if you can help to fix the code at http://play.golang.org/p/uOUjX5immw
References :
http://stackoverflow.com/questions/3676376/fetching-alexa-data
https://www.socketloop.com/tutorials/golang-http-get-example
By Adam Ng
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+5k Golang : Validate credit card example
+29.2k Upload multiple files with Go
+6.3k Golang : How To Use Panic and Recover
+14.8k Android Studio : AlertDialog and EditText to get user string input example
+12.7k Golang : invalid character ',' looking for beginning of value
+3.3k Golang : Constant and variable names in native language
+9.9k Golang : How to filter a map's elements for faster lookup
+5k Golang : Get environment variable
+5.9k Golang : Get curl -I or head data from URL example
+12.4k Golang : Find file size(disk usage) with filepath.Walk
+4.7k Ubuntu : connect() to unix:/var/run/php5-fpm.sock failed (13: Permission denied) while connecting to upstream