Our First AEM Rockstar Tips & Tricks Guest Blog!
Late last year we launched our first AEM Rockstar competition, where we asked for tips & tricks from you, the Rockstar AEM marketers and technologists who create and innovate within Adobe Experience Manager. Winners won a free pass to Summit and will present their tips & tricks to an audience of over 500 attendees and a panel of judges, who will award the best tip & trick winner with a cool prize!
Thanks again to all the Rockstars who competed this year. If you are at Summit and want to attend the Rockstar session, on Wednesday, March 22, click here: http://bit.ly/2kbauhJ! We’ll see you at Summit!
We were rocked by the response to the Rockstar contest, but ultimately we could only choose five finalists to present at the 2017 Adobe Summit. However, we received so many impressive submissions from the AEM community that we decided to feature a series of guest blog posts, written by the runners-up. These tips & tricks range from favorite ways to use Forms, to more technical examples that can help you better synchronize content, use new tools, and much more.
Our first guest post comes from Sagar Sane, who is a Technical Architect at iCiDIGITAL (www.icidigital.com), a digital agency that specializes in AEM. He has over four years of experience working with AEM / CQ5; primarily focused on server-side development and integrations of AEM with other systems, and over five years of web development experience. He is based at the iCiDIGITAL location in Raleigh, North Carolina.
Grabbit – A Rockstar Tool for Content Sync in AEM
Suppose you have an enterprise-scale AEM implementation with an author and multiple publishers in production. There is a staging environment mirroring closely to production from an infrastructure point of view, and you might even have development and UAT environments used for development and testing, respectively. The business stakeholders of your AEM implementation prefer that you keep the content synchronized as much as possible between these environments. Generally, you would use tools like Package Manager or vlt rcp to achieve this.
However, these tools do not scale well enough for large implementations where your AEM environments are geographically far away, or that have very large amounts of content to be synchronized. At the core, both these tools use the WebDAV protocol. WebDAV uses XML for serialization and deserialization and uses HTTP handshakes for every node that is synchronized between two AEM instances. This means that any latency on the network will hurt the content sync performance, in terms of time and space efficiency.
Grabbit: A Different Approach
The name Grabbit refers to this grabbing of content from one CQ/AEM instance and copying it to another. However, it also refers to Jackrabbit, the reference JCR 283 implementation that the content is being copied to and from. Grabbit is installed as any other AEM application and is available on bintray for anyone to download.
Grabbit takes a different approach to solve the problem of content synchronization. Grabbit’s goal is to make it easier and faster to sync content from one AEM instance (Grabbit Server) to another (Grabbit Client). Grabbit was developed as a content sync solution for one of our clients, with iCiDIGITAL members the primary contributors. Upon the success of the initial releases of the tool, Grabbit was ultimately open sourced by our client couple years ago.
How does it Work?
Unlike the other tools mentioned above that use HTTP handshakes for each node to be synced, Grabbit creates a stream of data over HTTP(s) from the Grabbit Server (source) to the Grabbit Client (destination). For serialization, Grabbit uses Google’s Protocol Buffers. Better than XML- based serialization, Protocol Buffers is extremely space efficient in transferring the data over the network. As its core underlying technology, Grabbit uses Spring Batch. Some of the core features of Spring Batch that Grabbit uses are Chunk-Based processing, Declarative I/O, start/stop a job and job processing statistics.
Grabbit needs to be installed on both the AEM instances; the instance that requests content to be copied to (the Grabbit Client) and the other one from which the content is copied from (the Grabbit Server). You initiate the request to copy content from Grabbit Server to Grabbit Client by providing a Configuration File to the client. For each path in the configuration file that needs to be synced, a new Grabbit Job is created on the Grabbit Client. Each job then opens only one HTTP(s) connection to the server and uses Protocol Buffers for data transfer marshaling / unmarshalling. There are several configuration parameters that can be set in the configuration file. You can find details about that here.
Grabbit also allows you to monitor the jobs on the client. The monitoring API provides basic information like the job start time/end time, current path, the number of nodes written, etc. Under the covers, Grabbit uses Spring Batch’s querying features for this. The job status is represented in JSON format as below:
To summarize, the basic sequence of Grabbit usage can be illustrated as below:
Why Use Grabbit?
Grabbit is supported for most of the newer AEM versions and is under active development. We have already received and continue to receive a lot of feedback to improve Grabbit. In our experience, Grabbit has proved to be 2 to 10 times faster than other content sync alternatives in AEM, depending on the network conditions and the amount of content being synchronized. I encourage the users of Grabbit to submit issues, questions, or comments on GitHub. We hope that you give Grabbit a shot and that it proves to be useful for everyone in the AEM Community. Post a comment if you have any questions!