Golang : Get Alexa ranking data example
Problem :
You want to get a website Alexa ranking and process the data in Golang.
Maybe for tracking a competitor ranking or tracking a publisher ranking where you put your advertising money. How to do that with a software instead of visiting Alexa.com all the time?
Solution : For this tutorial example, we will use the old Alexa API version(before they got acquired by Amazon). The API can be accessed at :
http://data.alexa.com/data?cli=10&url=domainname
For instance, a query of socketloop.com popularity with
http://data.alexa.com/data?cli=10&url=socketloop.com
will yield the following result :
<ALEXA VER="0.9" URL="socketloop.com/" HOME="0" AID="=" IDN="socketloop.com/">
<SD>
<POPULARITY URL="socketloop.com/" TEXT="291466" SOURCE="panel"/>
<REACH RANK="237320"/>
<RANK DELTA="+31735"/>
<COUNTRY CODE="US" NAME="United States" RANK="191742"/>
</SD>
</ALEXA>
NOTE : If you wish to extract more data from Alexa.com, you will have to use the latest API. See http://docs.aws.amazon.com/AlexaWebInfoService/latest/
Here you go! The Golang code to extract popularity ranking :
package main
import (
"encoding/xml"
"fmt"
"io/ioutil"
"net/http"
"os"
"strings"
)
func main() {
domainName := "socketloop.com"
resp, err := http.Get("http://data.alexa.com/data?cli=10&url=" + domainName)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer resp.Body.Close()
alexaData, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
// convert []byte to type io.Reader with strings.NewReader()
decoder := xml.NewDecoder(strings.NewReader(string(alexaData)))
// loop till we get REACH token
for {
token, _ := decoder.Token()
if token == nil {
break
}
switch startElement := token.(type) {
case xml.StartElement:
if startElement.Name.Local == "REACH" {
//fmt.Println(startElement.Name)
//fmt.Println(startElement.Attr[0])
fmt.Printf("%s has popularity rank of %s \n", domainName, startElement.Attr[0].Value)
}
}
}
}
Output :
socketloop.com has popularity rank of 237750
NOTE : I've tried to unmarshal the XML but was unsuccessful. Could it be UPPERCASE or too much attributes ? I don't know... seems like a bug to me or is there something wrong with my code? Let me know if you can help to fix the code at http://play.golang.org/p/uOUjX5immw
References :
http://stackoverflow.com/questions/3676376/fetching-alexa-data
https://www.socketloop.com/tutorials/golang-http-get-example
By Adam Ng
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+8.3k Linux/Unix : fatal: the Postfix mail system is already running
+6.2k PHP : Proper way to get UTF-8 character or string length
+30.1k Golang : How to redirect to new page with net/http?
+11.5k Golang : GTK Input dialog box examples
+15.2k Golang : Intercept Ctrl-C interrupt or kill signal and determine the signal type
+16.7k Golang : Get input from keyboard
+31.2k Golang : How to convert(cast) string to IP address?
+5.4k Golang : Detect words using using consecutive letters in a given string
+7.6k Javascript : How to check a browser's Do Not Track status?
+19.7k Golang : Convert seconds to human readable time format example
+8k Golang : Check if integer is power of four example
+7.2k Golang : Create zip/ePub file without compression(use Store algorithm)