Tag Archive: release

There’s been a lot of talk about WURFL lately and with good reason. On June 6, 2011 I launched ScientiaMobile along with my business partners, Luca Passani and Krishna Guda.  If you’re reading my blog you probably know that I am the author of the popular mobile device detection library Tera-WURFL.  I wrote Tera-WURFL back in 2006, when the world of mobile devices was completely scattered in every direction.  At the time, the vast majority of mobile devices did not support a common set of capabilities, like MP3, MMS or HTML.  Instead, your best bet was to write a terrible-looking, but fairly well behaved XHTML-MP site using very conservative tags.  WURFL solved this problem nicely by accurately matching a visiting user to a device profile that allowed you to serve a WAP page to a Nokia 3360, an XHTML-MP page to a Motorola RAZR and an HTML page to a 1st-gen iPhone.

Since 2006 there have been a lot of changes, particularly in the capabilities of mobile browsers.  Thanks to Apple Webkit, even lower-end phone are able to browse sites without catastrophic rendering issues (still a terrible UX, just not a catastrophe).  With all these changes, however, one thing has remained: mobile device fragmentation.  Device fragmentation means that the capabilities and methods of interacting with mobile devices are constantly spreading in different directions, none compatible with the other.  Technically this problem is driven by consumers, since we always want the next cool feature.  What happens is that manufacturers battle it out in an attempt to be the first to release a device that has an edge over the competition, like Near Field Communications and glasses-free 3D Displays.  The problem is that manufacturers release devices with these features before the industry has settled on a standard for interacting with them.

So what does this have to do with WURFL?  Luca Passani and I have seen the writing on the wall and we realize that this problem of device fragmentation has had some temporary relief by the massive market share by Apple and Android devices, but things are starting to get worse fast.  Every day we are seeing new devices and incomplete software features that make mobile web programming a nightmare!  For example, everyone is very excited about HTML5, but how do you know if the device you’re targeting supports the HTML5 features that you need?  One option is to use a high-level JavaScript abstraction framework like jQuery Mobile.  This is very slick on my super fast Snapdragon-powered HTC Inspire 4G, but how will it work on a typical off the shelf phone?  What about BlackBerry OS 6/7, MeeGo, Symbian^3, WebOS and WP7?  Good luck.

What Luca and I decided to do is to join forces and refocus our full attention on the problem.  In June we publicly announced the merger of the WURFL project and the Tera-WURFL project into a single company called ScientiaMobile (original WMLProgramming message).  Now that we’re on the same team, Luca and I are able to work more closely on the problem of device fragmentation and detection.  In addition to better APIs and more consistent data, we’re bringing some awesome new products to market that will make it easier for organizations of all sizes to get started with mobile device detection.  Right now, we are polishing up a product that has always been near the top of my feature requests:  WURFL Cloud.  WURFL Cloud is something that we’ve been secretly working on for some time and it is now in it’s Beta-testing phase.  We are spending a lot of time to make sure this service is very fast and accurate from day 1 (by the way, you can drop us a line at wurfl-cloud-beta@scientiamobile.com to get your Beta license).  Unlike the standard WURFL APIs, WURFL Cloud is a hosted device detection service that is always updated and maintained by the WURFL Team.

Throughout this process, we have chosen to follow our roots.  WURFL has always been a Free and Open Source project and it still is.  We value our community and are excited to bring WURFL to the next level while keeping it in the public domain.  At ScientiaMobile, in addition to the FOSS APIs, we are now selling commercial licenses and support for our APIs as well.  Under this new dual-licensing scheme we are able to help organizations that are not comfortable with the restrictions of the open source license.

As we say at ScientiaMobile: WURFL has Graduated!

After a lot of development and tons of regression testing, Tera-WURFL 2.1.4 is finally ready for release. There has finally been a break in commits on the GitHub repo for more that a week, and no bug reports on the development branch.

Changes from Tera-WURFL 2.1.3

  • Switched to better XML Parser (old parser, SimpleXML will still be used if XMLReader is missing)
  • Improved matching for Apple, Android, Nokia, BlackBerry / RIM, DoCoMo, KDDI, LG and OperaMini
  • Allowed preloading a custom TeraWurflConfig so the default is ignored
  • Updated source documentation
  • Updated instructions
  • Greatly improved cmd_line_admin.php
  • Converted line endings to \n
  • Various bugfixes
  • Improved Remote Client
  • Added builtin regression testing (thanks digitalronin!). Tip: from the shell, go to the test/ dir and type “rake sanity” to make sure your patches are compatible with the loaded WURFL
  • Moved default config to “TeraWurflConfig.php.example” so your config isn’t overwritten
  • Fixed MSSQL DB Connector bug

If you rely on Android or BlackBerry detection, you will really want to get version 2.1.4 – it will significantly improve detection of these devices.

I’ve been delaying the Tera-WURFL 2.1.3 release since mid-July because of the large number of feature requests and improvements coming in, but the time has come to release it. I’m still waiting for the unit test code to stabilize a bit before I put it into the tree, but if you are interested in it, follow me on GitHub and you should see it soon.

Here’s a short list of features and improvements in Tera-WURFL 2.1.3:

  • Native support for MongoDB
  • Better support and performance for Microsoft SQL Server
  • Command line administration utility
  • Better webservice performance using JSON
  • Better Python client with JSON support
  • Improved performance in MySQL5
  • Device Images are available via the PHP webservice client
  • Loaded WURFL version is now tracked in the DB
  • MySQL connector allows for alternate port
  • Better overall detection

If you’re a command line junky like me, I think you’ll really like the CLI administration tool.  To use it, just go to your Tera-WURFL/admin/ directory and type php cmd_line_admin.php

# php cmd_line_admin.php

Tera-WURFL Stable 2.1.3
The command line WURFL updater for Tera-WURFL
Loaded WURFL: www.wurflpro.com - 2010-09-09 04:08:06
Last Updated: Wed, 15 Sep 2010 20:38:59 -0500
Usage: php cmd_line_admin.php [OPTIONS]

Option                     Meaning
--help                    Show this message
--update=   The source of the WURFL file:
Update from your local wurfl.xml file:
Update from wurfl.sourceforge.net:
--clearCache              Clear the device cache
--rebuildCache            Rebuild the device cache by redetecting all
cached devices using the current WURFL
--stats                   Show statistics about the Tera-WURFL Database

You can update the WURFL by passing the —-update=local or –update=remote parameter (crontab will like this):

# php cmd_line_admin.php --update=remote
Downloading WURFL from http://downloads.sourceforge.net/project/wurfl/WURFL/latest/wurfl-latest.zip ...

done (/testtw/2.1.3/mongo/data/wurfl.xml: 14.80 MB [897.30 KB compressed])
Downloaded in 4.6609511375427 sec @ 1.58 Mbps

Database Update OK
Total Time: 22.146492004395
Parse Time: 3.8051941394806 (TeraWurflXMLParser_SimpleXML)
Validate Time: 0.024373054504395
Sort Time: 1.2780990600586
Patch Time: 0.50953578948975
Database Time: 8.9265999794006
Cache Rebuild Time: 7.6026899814606
Number of Queries: 2456
PHP Memory Usage: 54.49 MB
WURFL Version: www.wurflpro.com - 2010-09-17 15:50:54 (Fri Sep 17 15:56:32 -0500 2010)
WURFL Devices: 13687
PATCH New Devices: 39
PATCH Merged Devices: 1

You can also see stats on the Tera-WURFL database:

# php cmd_line_admin.php --stats
Tera-WURFL Stable 2.1.3
Database Type: MongoDB (ver MongoDB 1.6.2)
Loaded WURFL: www.wurflpro.com - 2010-09-17 15:50:54
Last Updated: Sat, 18 Sep 2010 15:28:05 -0500
Config File: /testtw/2.1.3/mongo/TeraWurflConfig.php
---------- Table Stats -----------

Rows:    13726
Devices: 6229
Size:    30.00 MB

Rows:    522
Size:    1.23 MB

Head over to Tera-WURFL.com and give version 2.1.3 a try!

I’m just about to release Tera-WURFL 2.1.0 (Stable) and I thought I’d give you an idea of what to expect. I was originally positioning this as a revisional update (i.e. 2.0.1), but decided to make it a minor version upgrade because it adds a lot of functionality and includes a few more settings than version 2.0.0.

Here’s what’s new:

1. Experimental support for Microsoft SQL Server 2005/2008: with the MSSQL2005 Database Connector, you can now use MS SQL Server as a Tera-WURFL backend! The support is experimental for now, although I’ve put it through my barrage of over 45,000 user agents and it works properly. The Reduction in String stored procedure still needs to be optimized since I just ported it from MySQL 5 to T-SQL. The MSSQL backend is considerably slower than MySQL, and I beleive it’s that procedure slowing it down.

2. SimpleDesktop Matching Engine: Tera-WURFL wasn’t originally designed to differnetiate between desktop and mobile browsers (either was WURFL for that matter), but with this release I’ve introduced the SimpleDesktop Matching Engine which, when enabled, uses keywords and regular expressions to detect 90% of desktop browsers without having to resort to searching the database for a matching WURFL entry. In my tests performance increased by 176% for detection of 45,000 actual unique user agents (both mobile and non-mobile). This feature also dramatically decreases the number of items in your cache by using a single cache item for all desktop browsers.

3. Capabilities Filter: I’ve been meaning to implement the Capability Filter for a long time, but two very high traffic clients of mine convinced me to sit down and get it finished. This adds a new setting in TeraWurflConfig.php called CAPABILITY_FILTER. If you set it to false it will be disabled and all the capabilities in the WURFL will be stored in the database and available to your scripts (this is the pre-2.1.0 behaviour). Here’s where the magic starts, you can set it to an array of the capabilities and groups of capabilities that you want to store and use, for example, if you just want the know what kind of device is visiting your site and whether or not it’s wireless, you can use this filter:

public static $CAPABILITY_FILTER = array(

This will shrink your device database by more than a factor of 10. I tested the filter with 20 capabilities against 45,000 unique user agents and it reduced the size of the cache from 645MB (without filtering) to 92MB (with filtering), then down to 24MB with both filtering and SimpleDesktop.

You can look forward to the Tera-WURFL 2.1.0 release around February 10, 2010.

Tera-WURFL is available at it’s usual location, http://www.Tera-WURFL.com/

Tera-WURFL started in late 2006 as a personal project to make the WURFL PHP Tools faster by storing the device capabilities in a MySQL Database instead of flat files. It’s now late 2009 and I’ve finally got Tera-WURFL 2.0 up and running. As of now I’ve released RC4 and I’m working on RC5. Hopefully I can get version 2.0.0 Stable out before I leave for Iraq. Here’s a breakdown of the features of Tera-WURFL 2:

Rewrote some of the UserAgentMatchers and deleted others to bring Tera-WURFL on par with the Java WURFL API. With the introduction of desktop browser UserAgentMatchers, we no longer need to use the large web patch; instead, you can just use the 8KB one from wurfl.sourceforge.net (included). Also, I fixed some typos and bugs here and there. NOTE: if you are upgrading from version 2.0.0 RC1-RC3 you should delete all your database tables before you update. You can leave the terawurflcache table if you want to retain your cache. Although it will still technically work even if you don’t delete the tables, you will be orphaning some unecessary tables in your database.

Complete code-rewrite from the ground up. The 2.x version of Tera-WURFL is loosely based on a pre-release of the Java WURFL Evolution Library, but the API is taken from Tera-WURFL 1.5.2. The following is a list of features found in Tera-WURFL 2.0:

  • User Agent Matchers have been created for each of the major manufacturers. These allow for specific matching methods to be applied to the user agent like string searching, RIS (Reduction in String) and LD (Levenshtein Distance).
  • Multiple patch files are now supported. Tera-WURFL ships with the current wurfl.xml, web_browsers_patch.xml and custom_web_patch.xml. Patch files can be added to TeraWurflConfig.php by separating them with semicolons in the TeraWurflConfig::PATCH_FILE directive. Patch files are loaded in order from left to right on top of the WURFL file, so if you want to override every other patch file, specify it last.
  • The custom_web_patch.xml file can be edited from the Web Administration page, and allows you to easily add non-mobile user agents to the patch file. The devices with these user agents will be detected as generic_web_browser (non-mobile).
  • Persistent Caching means that your cached devices stay cached. When you update the WURFL file or your patches, your device cache is also updated via the new database.
  • Cache Browser allows you to see what devices are hitting your site and what their capabilities were detected as.
  • Installation Script is better than 1.5.2. Once you download Tera-WURFL and extract it, edit TeraWurflConfig.php then go to /admin/install.php and follow the directions to finish installation.
  • PHP short_open_tags are no longer required to run Tera-WURFL. PHP has this feature disabled by default now.
  • Conclusive vs. Inconclusive Matching. If a device is matched with the UserAgentMatcher’s primary matching method it is considered a conclusive match, if it is detected via a recovery matcher or by the CatchAllMatcher it is an inconclusive match. This information is available via the tera_wurfl capability group.
  • tera_wurfl Capability Group. The TeraWurfl->Capabilities array now contains a group called “tera_wurfl”. This group contains the following Tera-WURFL related information:
    • num_queries – the number of database queries required to lookup the device.
    • actual_root_device – the WURFL ID of the actual device (not subrevision or generic), this can be null.
    • match_type – either conclusive or inconclusive.
    • matcher – the name of the UserAgentMatcher that detected the device.
    • match – whether or not there was an actual match. If there was no match, Tera-WURFL guessed which generic device is most similar to the device.
    • lookup_time – the time in seconds that it took to detect the device.
    • fall_back_tree – the complete fallback tree that built the capabilities of the device. This is a list of all the WURFL IDs from the detected device down to the base generic device.

Example Script

// Include the Tera-WURFL file
// Instantiate the Tera-WURFL object
$wurflObj = new TeraWurfl();
// Get the capabilities from the object
$matched = $wurflObj->GetDeviceCapabilitiesFromAgent();
// Show whether there was a conclusive match
if($matched){echo "Match found";}else{echo "Match NOT found";}
// Print the capabilities array
echo "<pre>".htmlspecialchars(var_export($wurflObj->capabilities,true))."</pre>";

Thanks to everyone who helped me test Beta 1.4.5RC1 – this pre-release was the stepping stone to version Stable 1.5.0.
I was lucky to get some help testing 1.4.5RC1 from a client that gets very high traffic. After upgrading from 1.4.4 to 1.4.5RC1, his server’s mysqld process went from 20-40% down to 6%!

Here’s a quote from him:
“The system gets over 2 million WAP hits per day … The impact on the server is definitely significant. Before this, mysqld was using between 20% and 40% of the CPU, and now it’s down to 6% or less. Load average on the box was averaging 3.84 before, and now it’s averaging 2.72.”

I actually released “Stable 1.5.0” a few weeks ago and since released “Stable 1.5.1” but apparently I never applied the changes to the main Tera-WURFL.com site so you wouldn’t have seen 1.5.1 in the downloads section. It’s up now so check it out! It features a very high speed caching system that caches the entire device capabilities tree for top performance.

I’m currently looking for suggestions for new features and improvements that I can include in future releases of Tera-WURFL. If you have something you’d like to see please let me know!

To download Tera-WURFL visit http://www.tera-wurfl.com/.

After many many hours hunched over my keyboard, I have finished a major revamp of the Tera-WURFL code. Here's what's new:

Completely rewrote the error logging system and verified it's operation after a bug was discovered (thanks Neil!)
Added many features to the web administration console – seriously – check out the online demo!
Changed default DATADIR to the included 'data' directory
Completely rewrote the README file to include detailed installation instructions and other useful info
Optimized the clean installation process – now Tera-WURFL has a brain!
Included the current stable release of the wurfl.xml file so you don't need to download it
Included database statistics, log file monitoring and global configuration in new web interface
Changed default log level to LOG_WARNING instead of LOG_ERR
Described in great detail the purpose of the different database tables

If you are using Tera-WURFL 1.4.3 or earlier I would highly recommend that you upgrade to 1.4.4 due to some fairly significant bugs!

Tera-WURFL Website

UPDATE 25Feb2008 – Adobe has published the recommended workaround for this problem.

This is another major release – I rewrote some of the code with some inspiration from Zelph.com’s onDOMload as suggested by Geoff Stearns, Author of SWFObject. I have now optimized the code to the point where all you need to do is include it in the head of your document and as the page loads, each object will be fixed, so by the time the page is done loading, everything is fixed automatically!

———EDIT 19Jan2007———
I found yet another IE bug related to this topic. After a page is cached by IE and reloaded, the SWF is loaded before it’s embedded, so any callback functions the the SWF tries to setup when it loads (like the JS->Flash ExternalInterface code) will fail with an error “objectID” is undefined. Then when you try to use the callback function you get Object doesn’t support this property or method because the functions didn’t get assigned to the Flash object properly.

To fix this error you need to put this line above your SWFObject code (or above your <object> tag):

window["objectid"] = new Object();

Here’s an example document:

You can download SWFFormFix2 Here:

You can see the nicely formatted and highlighted source code here:

I have patched the newest version of SWFObject (1.5) to include FlashFormFix 1.0.0. If you use this version in place of the standard SWFObject it will automatically apply SWFFormFix (bugfix for ExternalInterface() in a Form with IE) when you use it – this means all you have to do is replace your standard swfobject.js with the new swfobject_swfformfix.js and you’re done!

Here’s the optimized version:
(to download it right click and “Save Link As” or something similar)

Here’s the human-readable version in case you want to see what I changed:
(to download it right click and “Save Link As” or something similar)

I just finished SWFFormFix 1.0.0 – a BIG update from the 0.1.0 prerelease! This new version introduces Auto Mode which takes care of everything for you! All you need to do is include the script in you and call SWFFormFixAuto() before the closing tag!

For more specific instructions or to download it go here:

(to download it right click and “Save Link As” or something similar)

This version also contains an updated version of the previous function “SWFFormFix()” that is more optimized and probably faster than the Auto version.