Reducing repository size to improve performance
UPDATE: Please be aware that we no longer distribute Trie data files.
Historically, the Lite (free) versions of the 51Degrees data files have always been included in our open source Git repositories. As we update the Lite data file each month, this has, over time, led to our Git repositories becoming much larger than necessary, resulting in a poor experience for 51Degrees and third party developers alike.
Starting from 4th July 2017, 51Degrees will be switching to using Git LFS to store our data files. This removes them from the repository and replaces them with a placeholder text file. As long as the cloning system has Git LFS installed the data file will be seamlessly downloaded without any additional work on the developer's part.
In order to realise the performance improvements from the reduced repository size, we also needed to remove all instances of these data files from the history of the repository. After exploring a few alternatives, we settled on using the Git 'filter-branch' command. This allows us to remove all versions of the data files that have ever been committed to the repository.
In the end, this reduced the size of the repositories as follows:
- C - 548 MB to 1.6 MB
- C# - 1.08 GB to 92 MB
- Java - 1.33 GB to 28MB
(The cleaned C# and Java repositories are larger mainly due to some image files that are used for testing the image optimisation feature.)
Please be aware that these changes to the repository history mean that if you have existing clones of the 51Degrees device detection source code, those clones will now be incompatible with the new slimmed down repositories. If you have a clone with no changes, you can simply delete your local repository and re-clone the new one. If you have changes that you wish to keep whilst continuing to merge in the latest 51Degrees updates then you will need to create a new local repository, manually copy the changes to the new repository (using a folder & file comparison tool) and then delete the old repository.
Note: for those using the Trie detection method - We are now distributing the Trie files separately to GitHub and other repositories due to their large file size. Both the source code and Lite Trie data files are available in the forked public repository available to clone from https://git.51degrees.com/Device-Detection.git. The Premium and Enterprise Trie data files can be downloaded with a license key from https://git.51degrees.com/Trie/. If you have any questions please contact us.