Work on GSS, the open source cloud file storage platform, started in mid 2007. For years (2003-2006) we have been building custom web-based applications for storing, organizing and indexing documents. So, at the time, we already had various server side components, user interface designs & modules, a data model and a complete functional blueprint with rich semantics for a very promising browser-based file storage product. Unfortunately, being a services-centric company, we lacked the time (a.k.a funding) and the peace of mind between projects to actually build the product.Enter GRNET. The Greek Research Network (GRNET) decided to add one more innovative service to its portfolio offered to the Greek academic and research community: the vision was building an on-line Shibboleth-protected service that will allow users to store and share files. GRNET already had a mature cloud infrastructure and this new service would be hosted and grow in their cloud, elastically handling the expanding user community. To cut a long story short, we managed to win the RFP and started designing Pithos in tight cooperation with the very brilliant people from GRNET. By bringing in the additional semantics we have been working on for years (full-text indexing & search, fine-grained permissions, groups, etc) the project grew beyond the initial simple file storage service and the open source project GSS was born. At the time the buzzword "GRID" was still strong, thus the name GSS = Grid Simple Storage.
Growing up
As the first few thousand users started pouring into the Pithos service, development focused on stability and key features. Along with the on-line service Pithos, the GSS software started slowly to grow and mature. It was very rewarding to see your software grow organically as the real users discover new ways to use it and request features, identify bugs and point out bottlenecks. It was even more rewarding for us, since GSS was the first time we designed and delivered a system with a user group of more than a few hundred (actually the Pithos potential user community is more than 400.000 users).
Growth happened in bursts, following the availability of funding. This lead to long periods of relative inactivity and as time passed, building an on-line file storage platform started losing its 'hotness' value. On the flip side, slow aging (as in wine) sometimes helps avoid pitfalls and listen more clearly to the real user requirements.
Coming of age
Last November, after just having completed a contract from GRNet for upgrading Pithos, we realized that GSS has grown to be a mature platform with lots of potential. December 2010 (release 1.3) was an important milestone for the project, as we committed more resources to GSS and adopted it as an active, core EBS project. Since then we have been releasing frequently and greatly improved stability, performance and usability. Among other things we:
- upgraded to the latest Solr release and greatly improved indexing/searching performance (used Solr/Lucene in new ways, more to come on that)
- optimized the REST API in order to maximize client responsiveness and scalability
- optimized the back-end database layer (queries, JPA layer, etc)
- re-implemented various back-end layers for improved performance
- re-implemented the GWT web client for usability, responsiveness and less server interaction (improved browser caching and e-tags usage)
- implemented from scratch the WebDAV system interface, moving from Tomcat's implementation to Milton (more on that soon)
- added exposure of other users' shared folders via WebDAV
The future
Although we plan lots of core new features and updates targeted to specific clients (full GWT 2.2 web client, improved mobile clients for iOS and Android, desktop sync / backup), we also work hard on improving scalability and performance. Building on the experience derived from our first NoSQL implementation (gss2 branch) we plan to eventually release a much more scalable GSS that will be based on NoSQL, Solr/Lucene and a thinner application server layer. Adding distributed second level caching (currently working with JBoss Infinispan) is also underway. And being passionate JBoss fans, we cannot abstain from porting to JBoss 7 as soon as a final release is available, even though the MongoDB/Jetty implementation exhibited promising results.
