• About Us
  • Blog
  • Basket
  • Account
  • Sign In


Published on Tuesday, August 14, 2012

Gaining Device Statistics from W3C Server Logs

First, if you haven't checked out the latest release of the 51Degrees.mobi C API, you can grab it here and check out the documentation guide here. It's necessary to read through the whole guide before continuing with this post, as I'll be picking up from where I left off with the implementation of the isMobile function for parsing results strings. Make sure you can duplicate that before continuing.

There are a few challenges to address when extracting the required information from a W3C log file:
  1. Identifying the location of the UserAgent column within the log file.
  2. Extracting the UserAgent from the previously identified location.
  3. Compensating for the log file format by modifying the obtained UserAgent.
A working draft of the W3C log file format can be found here. Broadly, the log file contains a header of identification lines preceded by ‘#’ symbols before presenting the bulk of the data. The data itself is white-space separated into columns, which makes it easy to read and parse, but means that the data itself is less useful to use until some pre-processing has been performed.

First, a quick overview of the preprocessor commands for this implementation. This should go in your own .c source file or replace the text in TestFunctions.c:

#include "51Degrees.mobi.h"

Just to recap from the user guide, presented again here is the isMobile string parsing function that returns a string "True" or "False" depending on the "IsMobile" property of the device detected.
char *isMobile(char *result) {
    char *token;
    token = strtok(result, "|");
        while (token != NULL) {
            if (strcmp(token, "IsMobile") == 0) {
                token = strtok(NULL, "|");
                token = strtok(NULL, "\n");
                return token;
            else token = strtok(NULL, "\n");
            if (token != NULL) token = strtok(NULL, "|");
    return "null";

The solution implemented here uses the "strtok" function in the string.h header file. There is some controversy around using this function, mainly due to the fact that it modifies the source string, however, for the purpose of this implementation, it is more than adequate and can be easily replaced with an end user version if required. If you have already gone through the user guide, you will be familiar with it's use. If not, here again is the link to a description.

After implementing isMobile, we can take a look at the main function for our program:
int main(int argc, char* argv[]) {
    char *check, *result, subject[1500];
    FILE *input = NULL;
    int mobileCount = 0, nonMobileCount = 0, index = 0, i = 0;
    Workset *ws = createWorkset(); /*Create the 51Degrees.mobi workset*/
    /*Open file*/
    input = fopen("test.log", "r");
    if (input == NULL) {
        /*Exit program if file open failed*/
        printf("File open failed.\n");
        return 0;
    /*Identify the location of the user agent field.*/
    do {
        fgets(subject, 1500, input);
        check = strtok(subject, " \t");
    } while (strcmp(check, "#Fields:") != 0);
    do {
        check = strtok(NULL, " \t\n");
    } while (strcmp(check, "cs(User-Agent)") != 0);
    /*Perform the log check on each line of the log file.*/
    while (fgets(subject, 1500, input) != NULL) {
        check = NULL;
        /*Obtain the user agent from the subject line using the index identified earlier.*/
        for (i = 0; i < index; i++) check = strtok((check == NULL ? subject : NULL), " \t\n");
        /*Replace "+" symbols with " " from the userAgent.*/
        /*Populates the userAgent field of the Workset structure*/
        strncpy(ws->userAgent, check, MAXLENGTH);
        /*Get the device data from the user agent and scan it for the "IsMobile" identifier.*/
        result = detectDeviceFormat(ws);
        check = isMobile(result);
        /*Increment a counter based on the outcome of the "IsMobile" check.*/
        if (strcmp(check, "True") == 0) {
        } else {
    /*Output the total number of mobile and non-mobile devices.*/
    printf("Mobile results: %d\nNon Mobile results: %d\n", mobileCount, nonMobileCount);
    return 0;

Due to the flexible nature of the log file, the column quantity and location may change between servers. Because of this, it is necessary to scan for the header line denoted “#Fields”. Once located, the line should be scanned for the occurrence of the identifier “cs(User-Agent)”. By incrementing an index counter for each column we ignore while looking for this column name, we can easily find the location of the user agent in the body of the data. It’s worth noting that this technique works because empty columns in a log file must be indicated with a “-“, meaning there is no chance of accidentally skipping a column, assuming the file has been formatted correctly. This takes care of issue one on our list.

Issue two is now a lot easier considering we already used strtok to find the column name. By calling strtok multiple times without resubmitting a new string we can skip along a line until we find the correct location. Note the nested "if" statement within the function call. The first time we call strtok we must submit the string in question, subsequent calls while we examine this line must be null. Also, by submitting the newline symbol “\n” as a further token identifier, we deal with the quirk of the fgets function that cause it to return the appended newline character onto the string, which may skew our results.

So after taking care of issues one and two, we come to the final step of pre-processing that must occur before we send our UserAgent through the device detection algorithm. As mentioned previously, the log file format uses white-space separation to divide entries, therefore any spaces in those entries are plugged automatically with “+” symbols. A simple string parser that swaps these “+” symbols for “ “ has been included here which allows for an escape character for actual plus symbols that may exist within the User Agent string.

void changePlus(char* subject) {
    char *checker = subject;
    if (checker == NULL) return;
    while (*checker != '\0') {
        if (*checker == '+') {
            *checker = ' ';
        if (*checker == '\\') {
            if (*checker == '+') checker++;
        else checker++;
After performing these operations on the log file, we are now ready to copy the resulting userAgent to the ws->userAgent field of the workset and submit it to device detection, then finally to our string parser which will return “True” or “False” depending on the IsMobile flag. All that is left to do is increment the correct counters and repeat for each line of the log file.

All of the code presented here can be cut and paste into the “TestFunctions.c” file provided with our Lite distribution, then recompiled either with your development environment of choice or the 51Degrees batch file.

Finally, if you need a log file to test this program on, you can download one here.
Comments (0)

Author: Products Team

Categories: Development

Tags: C


Products Team

Other posts by Products Team
Contact author



.NET 2013 2014 4G 51Degrees 5G A.C.Roma A7 ABI Acer Adtech Affiliate Marketing Afilias Alcatel Amazon AMP Analysis Analytics Android Android 5.0 Lollipop Android Kitkat Android Lollipop Android Media Stick Apache API Apple Apple TV Archos Asha Asian Market ASP.NET Asus Australia Big Data Black Friday Blackberry Blink Browser C C# Case Study CeBIT CES Chrome Cloud CMS combinations Comparison Competition CoolPad COTW Cron CSS3 Data Data Blog Data File Data Model Daydream Denver Design Desire Eye Desktop Detection Developers Device Device Data Device Detection Device Intelligence Device Popularity Device property Device Types Device Use DeviceAtlas Display dmexco DoCoMo Doogee DotNetNuke Download Drupal Email EReader E-Reader Ericsson Evaluation Event Examples EXPLAY Rio Facebook feature Firefox Firefox OS Fly Foundation Framework France Galaxy S3 Galaxy S5 Galaxy Tab A Galaxy Tab A 8.0 Galaxy Tab A 9.7 Germany git repositories Global Google Google Daydream GSMA HAProxy Hardware Hisense HTC HTC ONE MAX HTC OS HTML5 HTTP HTTP Headers Huawei HUAWEI. UPDATE HUDL Huwaei IBC Icemobile Prime 4.0 IE IFA IIS Image Optimiser Image Optimizer India Infographic Ingeniux Internet usage iOS iOS 7 iOS 8 ipad iPhone iPhone 6 IsEmailBrowser IsWebApp Italy Japan Java Javascript Jolla Kentico Keynote Kindle Kindle Fire Kindle Fire HD Leagoo Lenovo LG Location Log File Analysis LTE Lumia m.dot macOS Map Memory Meta Data Mi 4S Micromax Microsoft Miia Style Mobile Mobile Analysis Mobile Analytics Mobile Devices Mobile Marketing Mixer Module Motorola MVC4 MWC MWC 2017 MWC16 MyPhone Native Native Apps NET New Release News News Letter Nexus Nexus 6 Nexus 9 NFC NGINX Nokia Nokia 3310 Non-Mobile NVIDIA Omate On7 OnePlus 5 Opera Opera Mini Operating System Optimisation OS OSX 10.10 OTA Panasonic Patent PC Pebble Performance phablet phone PHP Poland Presentation Press Release Price Band PRIV programmatic PS4 Publishers Python QMobile QR Codes Redirection Research Reseller Responsive Images Responsive web design RESS Review reviews RIM Ringmark RTB RWD Safari Samsung Scala ScientiaMobile Screen Screen resolution Screen Size SEO Server Server-side optimisation Set Box Set Top Box Sharepoint Shark 1 SHIFT phones Sitecore Sitecore version 9 SLUSH Smart TV Smartphone Smartphones Smartwatches Snapdragon Sony Sony Xperia Spain Swedish Beers Symbian Tablet Tablets Tesco Testing Tips Top 5 TOTW Tutorial TV UDS UK Umbraco Update updates US User Agent UserAgent User-Agent Vendors Version 3 Video VoLTE VR Wearable Web Web Apps Web content management WebKit WebMatrix White Paper Widgets Widnows WiFi Wiko Wileyfox Windows Windows Phone WURFL Xbox XBox One Xiaomi Xperia Xperia z Yosemite Z10 ZenFon 2 ZOPO ZTE