• About Us
  • Blog
  • Basket
  • Account
  • Sign In
  •  

Blog

Enhance Conference and the User-Agent

Published on Monday, March 7, 2016

Enhance Conference and the User-Agent

Misconceptions concerning User-Agents are limiting the possibilities

Friday 4th March 2016 saw 180 web professionals all with an interest in progressive enhancement assemble in London for a one day conference organised by Simon McManus.

Nat Buckley set the scene with an emphasis on low bandwidth, pure HTML and CSS web experiences with reduced reliance on JavaScript. Nat was the first of several speakers to quote Douglas Crockford a leading developer of the JavaScript programming language before explaining 0.9% of web browsers with JavaScript enabled fail to receive all the required JavaScript.

"The Web is the most hostile software engineering environment imaginable." - Douglas Crockford

Anna Debenham introduced the audience to the world of games console web browsing explaining they're all too often overlooked and highlighted the issues of comparatively weak hardware and browsers. Stefan Tilkov reinforced the theme of simplicity by refreshing the audience with the cycle of computing trends using CORBA and SOAP for reference. Forbes Lindsay advocated the server side approach used by Facebook picking React.js to demonstrate the art of the possible on both server and client web apps. Adam Silver used real eCommerce businesses to highlight how multi page checkouts resulted in more completed transactions than the single page JavaScript driven equivalents. Jen Simmons explained how CSS standards are now removing the need for more complex JavaScript alternatives providing the best quote of the day with "Don’t use bootstrap anymore". Other speakers covered themes of good copy, accessibility, design vs development and emerging technologies.

For my part I was impressed that so many qualified and respected speakers were endorsing the engineering principle of "keeping it simple" either to reduce complexity, improve user experience and even to increase business profit. As someone who often briefs designers and developers to avoid using JavaScript and particularly heavy frameworks I felt in good company.

Maligned User-Agents

However, I continue to be amazed at how the humble User-Agent is misunderstood. The User-Agent is part of the HTTP protocol and is designed to enable a client to advise the web server what it is. Such knowledge then enables a web server to adapt the content it serves to support the capabilities of the client including the physical screen size, available input methods, GPS support or software and browser capability among hundreds of others.

Jeremy Keith who facilitated the Q&A summarised the negative sentiment by "crossing" himself at the mention of "User-Agent". The consensus being that User-Agents "lie", were misused during the "browser wars", and as such can’t be relied upon.

It is therefore surprising to me at least that some of the speakers used data from Google Analytics and the browser information web site caniuse to support their arguments. These services all use User-Agents to determine browser versions, device type and the model of hardware.

Other than to highlight what a great job Simon McManus did arranging his first conference my purpose in writing this blog post is to reset some myths passed down from web professional to web professional over the years concerning User-Agents typified by debate and conversation at Enhance Conference.

Bad Experiences

Lumia 635 represents itself as an iPhone, Android and Windows Phone device!

The simple regular expressions and community maintained libraries which have been used to make sense of User-Agents aren't capable of providing reliable enough information for many business purposes. I believe the deficiencies of such solutions have biased most web professionals opinion on the subject.

Consider the following common User-Agent from Windows Phone 8.1 as an example.

Mozilla/5.0 (Mobile; Windows Phone 8.1; Android 4.0; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 635) like iPhone OS 7_0_3 Mac OS X AppleWebKit/537 (KHTML, like Gecko) Mobile Safari/537

Simply using a regular expression to search for the presence of iPhone or Android would lead to the conclusion that the device is an iPhone or Android device rather than a Lumia 635 running Windows Phone. The order in which the regular expressions are applied would impact the conclusion. Incidentally Microsoft have adopted this approach to User-Agent construction to counter web sites which use poor quality User-Agent detection solutions.

Feature detection, the process of querying a web browser using JavaScript to determine if it is capable of performing a certain function, provides more reliable results when compared to poor quality User-Agent sniffing.

Quality Device Detection

51Degrees, the business I started and run, maintains a device detection solution that provides a +99.9% reliable detection rate across a unique set of real world User-Agents and includes indicators concerning false positives. User-Agents are used alongside other HTTP headers and JavaScript when available to provide the most complete understanding of a device and its accompanying browser and operating system. When considered over a large enough volume of web requests it’s more reliable than the JavaScript figure provided by Nat Buckley, or the anecdotal problems associated with JavaScript implementation and support presented during Enhance Conference.

Distraction

The previous example shows how the Lumia 635 User-Agent is intending to deceive inferior detection solutions. 51Degrees eliminates these distractors. Use the following link to see the results from 51Degrees purely based on this single User-Agent input.

Mozilla/5.0 (Mobile; Windows Phone 8.1; Android 4.0; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 635) like iPhone OS 7_0_3 Mac OS X AppleWebKit/537 (KHTML, like Gecko) Mobile Safari/537

White Lies

iOS Request Desktop Site Icon

Of course the User-Agent may not always be correct. Rather than this resulting in a bad user experience it results in the user receiving the experience they requested. That’s why mobile web browsers have the option to request a "desktop site" which among other things alters the User-Agent provided to the web server.

Damned Lies

Sometimes irresponsible device and software vendors do not set their User-Agent header correctly. Sony’s Playstation Vita is a possible example highlighted by Anna Debenham in her excellent console analysis of February 2014. The browser is reported in the User-Agent as Amazon's Silk and considered incorrect.

Mozilla/5.0 (PlayStation Vita 1.69) AppleWebKit/531.22.8 (KHTML, like Gecko) Silk/3.2

Note: Amazon elude to the fact they provide a browser for the Vita in their advice for detecting Kindle devices based on the User-Agent. Vita 3.57, the most recent version of the Vita software, continues to report the browser as Silk nearly 2 years after the original analysis. I've not yet been able to recheck the Vita but will aim to do so in the next few weeks.

Of importance is that the hardware is correct and should the browser be incorrect this fact can be updated in the database of devices, operating systems and browsers.

The use case for device detection is generally concerned with identifying the hardware’s capabilities rather than the browsers. Feature detection should be used in addition to pure device detection for the most robust deployments.

Conclusion

I’m biased as I setup a company to catalgoue all web enabled devices, operating systems and browsers indexing the data by HTTP headers, native SDK values and TAC codes. 51Degrees now employs 22 people the majority of whom work with vendors to classify over 170 new devices each week with over 99.9% accuracy. Over 415,612 combinations are tracked. We don’t screen scrape GSM Arena or use regular expressions, XML and JSON. The underlying source code is licenced under Mozilla Public Licence 2 and available on GitHub. We have free versions of the database updated monthly, and weekly and daily subscriptions which cost less than a typical developer spends on coffee.

When used in commercial businesses the results are clear. That’s why Wiggle, eBay, Microsoft, Boden, HSN and 1000s of others use the data subscription to analysis and optimise their web presence.

Read Case Studies

What to know more?

If you’d like to know more Martin Beeby and Martin Kearn of Microsoft and Enhanced Conference sponsor and attendees produced an excellent video on the subject which can be seen at Channel 9. I apologise in advance of the audio quality.

Ready to Try?

If you’d like to try the full product contact us quoting Enhance Conference and we’ll send you the full data set updated daily. I hope to see you at the next Enhance Conference event.

Try Us Now
Comments (0)

Author: James Rosewell

Categories: Opinion

Tags: User-Agent

James Rosewell
>

James Rosewell

Other posts by James Rosewell
Contact author

Name:
Email:
Subject:
Message:
x

Tags

.NET 2013 2014 4G 51Degrees 5G A.C.Roma A7 ABI Acer Affiliate Marketing Alcatel Amazon AMP Analysis Analytics Android Android 5.0 Lollipop Android Kitkat Android Lollipop Android Media Stick Apache API Apple Apple TV Archos Asha Asian Market ASP.NET Asus Australia Big Data Black Friday Blackberry Blink Browser C C# Case Study CeBIT CES Chrome Cloud CMS combinations Comparison Competition CoolPad COTW Cron CSS3 Data Data Blog Data File Data Model Daydream Denver Design Desire Eye Desktop Detection Developers Device Device Data Device Detection Device Intelligence Device Popularity Device property Device Types Device Use Display dmexco DoCoMo Doogee DotNetNuke Download Drupal Email EReader E-Reader Ericsson Evaluation Event Examples EXPLAY Rio Facebook feature Firefox Firefox OS Fly Foundation Framework France Galaxy S3 Galaxy S5 Galaxy Tab A Galaxy Tab A 8.0 Galaxy Tab A 9.7 Germany git repositories Global Google Google Daydream GSMA HAProxy Hardware Hisense HTC HTC ONE MAX HTC OS HTML5 HTTP Huawei HUAWEI. UPDATE HUDL Huwaei IBC Icemobile Prime 4.0 IE IFA IIS Image Optimiser Image Optimizer India Infographic Ingeniux Internet usage iOS iOS 7 iOS 8 ipad iPhone iPhone 6 IsEmailBrowser IsWebApp Italy Japan Java Javascript Jolla Kentico Keynote Kindle Kindle Fire Kindle Fire HD Leagoo Lenovo LG Location Log File Analysis LTE Lumia m.dot Map Memory Meta Data Mi 4S Micromax Microsoft Miia Style Mobile Mobile Analysis Mobile Analytics Mobile Devices Mobile Marketing Mixer Module Motorola MVC4 MWC MWC 2017 MWC16 MyPhone Native Native Apps NET New Release News News Letter Nexus Nexus 6 Nexus 9 NFC NGINX Nokia Non-Mobile NVIDIA Omate On7 OnePlus 5 Opera Opera Mini Operating System Optimisation OS OSX 10.10 OTA Panasonic Patent PC Pebble Performance phablet phone PHP Poland Presentation Press Release Price Band PRIV PS4 Python QMobile QR Codes Redirection Research Reseller Responsive Images Responsive web design RESS Review reviews RIM Ringmark RWD Samsung Scala Screen Screen resolution Screen Size SEO Server Server-side optimisation Set Box Set Top Box Sharepoint Shark 1 SHIFT phones Sitecore SLUSH Smart TV Smartphone Smartphones Smartwatches Snapdragon Sony Sony Xperia Spain Swedish Beers Symbian Tablet Tablets Tesco Testing Tips Top 5 TOTW TV UDS UK Umbraco Update updates US User Agent UserAgent User-Agent Vendors Version 3 VoLTE VR Wearable Web Web Apps WebKit WebMatrix White Paper Widgets Widnows WiFi Wiko Wileyfox Windows Windows Phone Xbox XBox One Xiaomi Xperia Xperia z Yosemite Z10 ZenFon 2 ZOPO ZTE