The G11n Innovation and Technology Summit 2017 was held at the Adobe HQ in San Jose on February 9th. The planning committee started with the vision to host an event whereby Adobe business leadership would discuss the steps Adobe is taking to increase revenue in international markets. To get a pulse of the industry, globalization thoughts leaders from Google, Microsoft, Intuit and SalesForce were invited to share their thoughts and global vision for their respective organizations.
The registration was open to all Adobe employees and our sessions quickly became filled. The audience included engineers, product managers, program managers and customer engagement teams from across Adobe offices worldwide.
The summit was dedicated to our dear colleague Warren Peet, who in his engineering manager role was a pillar of our Globalization team and one of the longest tenured employees in the company. He will be deeply missed.
The summit was dedicated to our dear colleague Warren Peet, who in his engineering manager role was a pillar of our Globalization team and one of the longest tenured employees in the company. He will be deeply missed.
| Attending the Summit |
The day started with an interesting keynote from Ajay Pande, VP, Engineering, Cloud Technology, Adobe. He talked about the ability to start at the developers’ code base, localize it with the vendor, and ship for all markets along with the English release. The way we at Adobe are striving to get the customer experience right is by running various experiments and have the global products change incrementally at a faster pace than ever before. He focused on using deep learning and related technologies to use data to get to a level of accuracy and correctness much more than ever in the past.
Ajay then handed it over to Macduff Hughes, Engineering Director, Google Translate, Google.He discussed the transition of Google translate from Phrase-based Translation to Neural Machine Translations.
There were two plenary discussions focusing on internal and external trends. The internal panel titled “Leaders’ Speak” comprised of :
They talked about what can be done to enhance the global customer experience and what it means to expand international outreach and business.
Meanwhile, the external panel was titled “TED-G” – the panel discussed the top challenges faced by their companies and innovative business models built to meet those challenges in the international markets. Panelists also touched upon topics like, Compliance, Regulations, Market specific features, scalability, Analytics and various best practices. The external panel comprised of :
These were followed by interesting demos and discussions hosted by Globalization engineers. The topics included:
• Advanced NLP services Keyword extraction, Categorization, Named entity extraction, Wikification (entity linking with Wikipedia)
• Multilingual text analysis Language detection, Language analyzers for processing multi-lingual text in various languages
• Machine Learning/Deep Learning based solutions Sentiment analysis, Spam detection, Semantic similarity, Auto-tagging text using multi-label classification techniques
• Augmented Reality
| Reflecting on the Summit |
After the summit, we received some interesting quotes and feedback from our attendees and speakers:
“The panel discussion showed the passion we all have for our global customers.It was also a reminder that each of us must question the value of the work we’re doing for our customers.If we’re translating content that isn’t used, we have to question how our resources could have been better spent to help customers succeed.” – Chris Hall
“It was great to attend the Globalization mini-summit this year. The planning and content of bringing not only people in from across Adobe to speak to various issues, but having external speakers come to talk about their companies and experiences was a brilliant idea.It made for very interesting sessions.” – Priscilla Knoble
“The summit was a good chance for me to share how Japan teams work with other teams to support local business from G11n perspective, also had a good interaction with other leaders and attendees to discuss how we should work together to achieve Adobe’s strategic goal in coming years. At the same time I learned a lot about how other companies are working on G11n, what are their challenges, how they deal with the issues, etc. “ – Xiang Zhao
For questions regarding this article, please contact author Akulaa Agarwal at email@example.com
An Open Letter to Adobe Translation contributors and subscribers
A few years ago, we envisioned that the Adobe International Community would like to be involved in improving the quality of the products they use. We built the infrastructure that enabled our community to freely contribute feedback, vote on translations, propose new translations, and create new language offerings for some products.
While quality work is never “done”, we feel that we have achieved many of our objectives. Now is the right time to reimagine how we should engage with our Adobe community to support international releases in an agile world, where innovation rules.
On 24 February 2016, we closed the Adobe Translation program and took down the site (ref. https://translate.adobe.com/adobe). We would love to receive feedback about your experiences; hear your suggestions for the future; and ideate with you about how to involve the Adobe international community in improving our products.
We give heartfelt thanks to you, our generous international community, for supporting this translation initiative over the years. You have lent your time and talents and shown sincere dedication. For that we are indebted and grateful.
The blog talks about the importance of automation in the world of localization and its increased need in the world of Agile
Paradigm Shift from Waterfall to Agile in the World of Localization
Do you know which is the fastest land animal in the world reaching speeds up to 113km/h?
Over the last two years, there’s been a gradual shift in the Software Development Life Cycle (SDLC) methodology for most of the Adobe flagship products. Product management has moved from yearlong waterfall product development life cycle to sprint-based Agile methodology (based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams).
As a result of changing market trends, we need to reinvent our approach to localization testing to meet the changing requirements of Agile methodology. In Agile, development and test cycles are shorter with quick turnaround time. In localization, test volumes have spikes considering the duration of the sprint cycle of 2-3 weeks. Features require frequent validation across multiple locales and platforms before certifying for a release in a simultaneous release model (sim-GM). In the Agile framework, it’s important to be cognizant of the business goals from localization perspective. I would categorize these in three broad areas:
Time boxed frequent releases to market: With agile most of the Adobe products have at least one release every quarter to a frequency as high as weekly/monthly releases
Increased test scope leading to increased localization efforts: With each sprint the legacy scope to certify LOC build increases
Higher focus on rapid new feature development with simultaneous release to market: Certifying features on N Locale and M platforms in a sprint of 3-4 weeks
These goals create the following challenges for an International Quality Engineering (IQE) team while deciding the scope on the localized builds for localization testing:
Ensuring increased test coverage on the new features while balancing the coverage for legacy feature areas
Ensuring re-usability of tests across various platform variants
Ensuring test accuracy across repetitive scenarios on multiple languages
Ensuring faster execution to uncover defects early on
Ensure all the features work as expected on all supported platforms and locales
Ensure co-existence with different versions of the released product/patches
Ensure shipping the product simultaneously in all supported locales across geographies (sim-GM)
Ensure optimized test coverage on all the supported locales and platform variants
Automation in Agile & Localization
Why automation testing in Localization?
– Multiple Releases in an year
– High volume of testing
– Complexity of Platform: Locale combination
– Improved test coverage leading to better quality
– Faster time to market
– Cost Effectiveness
With these initial thoughts, we proposed to expand the automation coverage of Dreamweaver product features from English language to localized languages in September 2012. Our initial Goal was to attain 45% feature automation coverage on localized builds with respect to coverage on English build on Mac platform.
Gradually we built the feature automation capabilities in the next six months, starting from enabling the automation framework for localization (i.e., added support to the automation framework to run scripts on the localized operating system) to running daily smoke tests on all the 15 supported languages, and eventually having good feature level automation coverage.
Automation is a great way to overcome the above challenges and effectively achieve optimized test coverage on localization builds. With automation, it would be possible to certify incremental creative cloud releases for all the supported operating systems and language combinations supporting the time-bound releases.
With multiple releases to market in a year, manual execution of the repeatable test scope by the localization vendors leads to increased test efforts. The major part of the increased test effort can be attributed to incrementally increasing legacy test scope, i.e., legacy scope is cumulative the sum of all the deliverables in the past of a product and would increase with each sprint. On the other hand, automated tests can be run over and over again ensuring defined coverage across platforms and language combinations, thereby contributing to the overall product quality for the time boxed release. This not only eliminates the test redundancy but also helps in producing faster test results.
Having the legacy area automated would help the localization tester focus manually on the current sprint deliverable, hence uncover defects early in the test cycle.The IQE needs to be cautious in deciding the scope of automation on localized builds. Prioritizing the automation coverage is very important.
With each quarterly release to market, the certification scope of the legacy features set for a product is increasing, leading to amplified repeatable test effort across multiple sprints compared to one time validation in the yearly releases model
Journey into Dreamweaver Automation
DW Localization team has 88% Functional Coverage & 86.5% of conditional coverage w.r.t core coverage of 50% conditional and functional coverage in CC release for MAC!
For adopting product automation in localized languages, our journey stared by answering a few of our initial questions:
What features do we need to automate?
What will be the sequence of feature automation?
What locales should we consider to start with, based on data from prerelease and bug history?
What would be the best approach for optimized test coverage in the different locales?
In the automation framework, where should the locale specific strings (used in Test scripts) be placed? Or should we pull the strings directly for comparison from the Adobe Localization Framework (ALF) at runtime?
How much effort is required for adopting automation on locales?
What would be the initial setup required to start automation in the different locales?
How much additional effort is required for running automation scripts in localized builds regularly?
What will be the hardware needs and the challenge to meet them?
What should be the frequency of automation runs (daily smoke, basic feature plan, and new feature plan)?
How to have the best execution turnaround time on all locales? What should be the optimization matrix considering fast turnaround time in agile?
Initial 6 months journey into adoption of automation framework for Dreamweaver localization
Dreamweaver Automation Framework
Dreamweaver automation is based on the Adobe homegrown automation framework called ‘Jerry’. The framework was developed by the Dreamweaver English QE team. It was written in Core Java, supported by apple scripts and java scripts in the backend, making use of the Dreamweaver’s API exposed by the developers.
The diagram depicts the automation workflow:
Step 1: A job ticket (contains configuration details like TC #, Platform, Machine details, language information etc.) is fed into the Pulpo server.
Step 2: Pulpo server primary purpose is machine management and result logging. Pulpo server invokes the test machine based and executes the test automation based on the plan mentioned in the job ticket.
Step3: Once the execution is completed the log/results are copied to the Pulpo server for further analysis.
Step 4: Results are logged to the automation dashboard “AutoDash”
The Jerry framework contains automated test cases grouped under various test plans:
Daily Smokes – Basic test for validation of daily build
Basic Features Plan – Contains test cases of the legacy and new areas covering feature smoke in Test Studio
Acceptance Plan – Contains acceptance and full test pass coverage for features developed in legacy and present release cycle in Test Studio
We started with one iMAC machine dedicated to Dw automation. However, soon after proof of concept was successful, we added one more dedicated machine for automation on localized builds. The above test plans got executed on a pre-scheduled basis across all 15 locales on the predefined execution plan. Job tickets distributed across 15 locales were fed to the Pulpo server either manually or automatically and were triggered on the arrival of new build in Codex. Typically, by the time we arrived at the office, build sanity was completed on all the locales and we were good to share the builds with our vendor partners.
For monitoring and optimization of test coverage across 15 languages, a dedicated execution calendar was followed. Based on the calendar, different automation test plans were executed on various locales/platform combinations on a daily basis. Daily smoke test for build validation were executed, followed by dedicated full feature test pass on the weekends. The execution was pre-scheduled and the test coverage was distributed across locales for optimal results given the time and machine constraints.
Accomplishments & Learnings
In the Creative Cloud (CC) release, we benefitted from having automated test passes on localized builds across 15 languages:
Overall test coverage efficiency improved four folds compared to manual test execution
Quick sanity test for acceptance of localization build before passing the build to vendor partners increased efficiency
Achieved quick turnaround time for basic feature testing by automation scripts
Parallel certification on multiple builds (Patch and Main line builds)
More focus on new features development part of the current sprint by the localization functional testers
Prerelease build certification completely through automation
Built blueprint and Code Sign verification through automation on all locales in 2 hours compared to 32 hours of manual verification
Support from the core team: It is essential to have the automation blessed from the English team for optimal support. In case of Dreamweaver, we got immense support from the team, especially from Kiran Patil (Quality Manager) and Arun Kaza (Sr. Quality Lead) for driving the automation efforts on localized builds
Pilot on automation framework for one Tier 1/double-byte/Cyrillic locales to ensure the framework was robust and would support automation on most of the locales
Always document the issues /challenges you face during setting up automation, they always act as a reference point later
Ensure core scripts are independent of English strings. In Dreamweaver, updating the legacy automation scripts to make these scripts run on localization was a big challenge, as automation scripts were failing at string comparisons. Aishvarya Suhane (Localization Automation QE) was a great help for writing functions in automation framework and creating a few new scripts for resolving localization-specific issues.
Cheetahs in the world of localization …
Special thanks to Guta Ribeiro for inspiring & mentoring me to write my first Blog & Rakesh Lal for his support.
The 3rd Multilingual content special interest group meetup is around AEM multilingual DAM. About half of the community implemented AEM DAM. But DAM is a popular topic in that people are already asking about best practices around organizing, localizing and searching global assets. So this SIG is gearing towards addressing the need for those who want to know why DAM is needed and how other people are using it, as well as for those who implemented DAM but not sure if the way they use it is the best way. It also turned out to be very useful for our technology partners who are looking for customer use cases to develop DAM connectors. AEM customers from Intel, Xlinx, SAP, SuccessFactors, Sas Institute, Paypal attended, along with partners from Cloudwords, TDC, ClayTablet, etc.
The meeting started with an overview of AEM DAM given by Sr product marketing manager Elliot Sedegah. It was a very informative and interactive session with lots of demos. He demoed some of the AEM5.6.1 and 6.0 features such as collaboration workflow between creative professionals and marketing manager collaborating on asset revision, upload, etc. He also showed the new Scene7 DAM integration feature on dynamic text rendering on an image. Very cool stuff. Nick from Adobe.com later showed a POC on asset localization using this feature OOTB. Elliot also showed Asset Share which is kind of like a portal for asset search and download. This segways into a topic that many SIG attendees are interested in – what’s the best practice of organizing DAM assets. Many of them use folders. Elliot mentioned both using folders and using metadata. The advantage of using metadata is that it is part of the asset regardless of where it is. It’s also great for search. Most of these demos are on the touch-optimized UI, which is very cool looking.
Then Web Marketing Sr. Manager Gary Gamitian from SuccessFactors presented their DAM use case on SuccessFactors.com. They launched CQ5.5 last December and so far has several language/locale sites. Gary showed us their Resource Center site. It uses both tagging and metadata driven search for their digital assets. They have very strict rules on metadata reinforcement. One thing he mentioned what would be a useful feature is the language copy functionality for DAM assets. That would automate target asset population.
The afternoon session is around asset localization. Sr. Web Production Manager Nick presented how Adobe.com will leverage the new Scene7 feature for asset localization. Basically the integration allows one to select an asset directly from the Scene7 repository through AEM. The text layer on the image is parameterized so you can change the text directly on the website component on the fly, or package it up as translatable content to send to a Translation management system. Sara Lockhart-Sirman, Web Operations Manager at Intel also show cased how they are using the AEM customized component to localize text on an image on CQ 5.4.
Finally, Sr. Product manager Cedric Huesler and Sr. Translation Technology Group Manager Chris Duran gave an update on the MSM issues this community reported. I saw some happy faces in the room on this one.
Great sessions overall, and the refreshment is awesome too!
Adobe Captivate is an electronic learning tool which can be used to author software demonstrations, software simulations, and randomized quizzes in swf and HTML format which can be converted and uploaded to video hosting websites. This content can be shared over Facebook and Twitter to make eLearning a very simple and interesting task. Adobe Captivate is shipped in 7 locales – English, French, German, Japanese, Spanish, Korean and Portuguese but courses and demonstrations can be created in other languages as well and we can share these localized courses very easily. Getting an online content in native language sets learners free of their dependency on English locale.
Creating localized eLearning courses
Below steps guide that how simple it is to share your high quality creations and demonstrations on YouTube and further over twitter/Facebook via Adobe Captivate 7 without even having much prior knowledge of the product. It will also highlight some of the trivial yet important issues which might prevent users to share content in English as well as non-English locales. This solution can be helpful to many native content creators.
1. Launch Captivate and select Video demo or Software Simulation from Start up page. Software simulation records events such as mouse click, keyboard entry, and system events and create slides accordingly. Video demo lets you create a single video which can be directly published to mp4 file.
2. Select the application or screen area which you want to demonstrate and select default presets for idevices(iphone, ipad), YouTube or customize it as per requirement. Select panning mode to focus screen areas manually/automatically with mouse movement.
3. Select “settings” button and go to Global Preferences. And select the language in which you want to generate the captions. Captions are generated automatically in case of software simulations which help in guiding throughout the training. Captivate allows you to create captions in multiple locales present in the list. They can be edited manually if required.
Captions can be generated in any of the languages selected by the user.
4. Add audio narration to your demo by selecting proper audio input device. System audio can be added as well to the projects along with narration.
5. After all settings click record button. Add narration and keep demonstrating your project. After completion hit END or click system tray icon in task bar.
6. The video will play before you and there is option to edit it but if it is properly recorded just click “YouTube”. Click the Folder icon to publish it locally as mp4 file. (This can be shared as standalone file as well) . You can adjust the aspect ratio, quality, FPS in this workflow. Even after getting a copy store at your machine captivate asks about YouTube publish as well.
Edit Mode: If the video needs some correction click “Edit” and modify the video. Cutting of extra video, zooming on important areas, split of video into two parts, inserting objects, inserting another PIP (picture in picture) video is possible in edit mode.
Sharing courses on YouTube, Facebook & Twitter
1. Publish to YouTube:
If you already have a YouTube account, enter your credentials and accept the license agreement and Log in to YouTube.
For new users with no YouTube account, click over new user and you’ll be redirected to sign up page for Google. Create your account and after successful creation come right back to Adobe Video Publisher and enter your details. Many a times users face an issue that even if after successful creation of account they are not able to login and face an error that specified user name/password incorrect although on YouTube they can log in with same credentials.
The issue is that every new account on YouTube needs verification. Without account being verified you cannot upload your video. To verify the account you can create your YouTube channel or verify via your phone.(This applies to Adobe Presenter Video creator as well)
Once you have logged in add description to your video and mark it under proper category public/private and UPLOAD! You can further view you video on YouTube or copy link to share with peers.
2. Publish to Facebook and Twitter
To share the course on Facebook/twitter check respective buttons and POST.
Sometimes user might face a blank dialog or a dialog saying internal error occurred when they post video on Twitter and issue is not easily isolated. Even restart/republish does not lead to any success.
The reason behind this is oAuth issue. We have to ensure that system’s time stamp is in Sync with Twitter’s. Twitter returns the current time in the “Date” HTTP header with every request. If your request fails due to a time stamp mismatch, use this time to determine the delta between the system clock and its server clock and adjust your oAuth_time stamps for subsequent requests accordingly. So all you need to check is that system time is set same as per proper time zone and you have not modified it. For eg. If your time zone is (US and Canada) Eastern Time then your system time should be same as current time of that region only else twitter won’t return your request. And you’ll get internal error. This issue can occur with people following different locales who may set their zone to some other region and system time to their own region. Can refer https://dev.twitter.com/discussions/204for more details.
(Also applicable to Adobe presenter video creator.)
By following the above simple steps, students can now use their own Facebook/Twitter accounts to learn more by enrolling in these courses. It will be a great experience to learn via social networking portals . So start recording and sharing your projects in ADOBE CAPTIVATE and share them worldwide to connect with more and more people and encourage eLearning – the most fast and efficient learning practice.
Over the past decade, many software development teams have switched their development methodology from a waterfall model to something much more agile such as Scrum. Through this transition, their expectations towards other teams such as Localization have changed and these teams have had to improve their agility too.
At Adobe, we have a centralized Localization group that currently supports 135 product and functional teams. Most of these teams have adopted some form of agile development methodologies and have reduced their development cycle from 18-24 months to yearly, quarterly, monthly and, these days, bi-weekly releases. A couple of Adobe product teams and other companies are even releasing updated versions of their product multiple times per day, making it imperative for Localization to keep improving its agility.
Drawn from our experience, this article presents Five Golden Rules that need to be satisfied in order to achieve optimal agile localization.
Rule 1 – “We are one Team!”
Within many companies, Adobe included, Localization is a centralized function serving all product and functional (e.g. Marketing, Sales, Legal, etc.) teams. This structure makes sense because localization is a specialized field, therefore resources (people, tools) and processes can be leveraged across the company. Nonetheless, localization should still be everyone’s concern. The Localization team can come up with many solutions, but the best ones originate when there is a true partnership between the product/functional and localization teams.
The most agile teams treat Localization staff as if it were part of their own team.
A core aspect of Scrum is to include all skill sets, including localization, required to deliver a product to users. Therefore, Localization team members must be included in all development aspects – from backlog review to retrospective – so they could plan and address international issues early on. Strong partnerships also need to be established with localization vendors when companies, such as Adobe, engage with partners and vendors for their translation and testing activities.
Customer engagement is a key aspect of agile methodologies as it validates the quality and usefulness of the work performed thus far. We recommend engaging with international customers too, because their issues increase awareness around internationalization.
In summary, all stakeholders (development teams, functional teams including localization, vendors and customers) need to collaborate closely in order to achieve great agility.
Rule 2 – “Internationalization is King”
In our Globalization Myth Series, we defined Internationalization (commonly abbreviated as i18n) as an engineering exercise focused on generalizing a product so that it can handle multiple languages, scripts and cultural conventions (currency, sorting rules, number, date and time formats…) without the need for redesign.
In other words, the better internationalized an application is, the easier it will be to localize.
In the waterfall model, teams could possibly work around some of the internationalization deficiencies because of longer development cycles. Unfortunately, in the agile world, there is not enough time to look for work-around solutions anymore. The code needs to be internationalized from the get-go.
There are various approaches to improve internationalization in a company, which includes the following:
Education: Training core developers is an effective way to reduce the number of internationalization issues in a product. By exposing engineers to localization and internationalization issues, they gain a broader perspective on the impact of their code and avoid some of the classic internationalization pitfalls.
Code review: Practicing peer reviews is an effective method to reduce internationalization defects in a product. Since developers know their code will be reviewed, they pay more attention to its quality (peer pressure effect) so it also benefits internationalization. Some companies even automate this review process using tools such as Globalyzer.
Globalization Report Card: Benchmarking products against an ideal architecture helps to improve internationalization too. As part of our World-Readiness program, we created a Globalization Report Card system to assess the degree of world-readiness in each Adobe product. This scorecard measures products against a set of internationalization criteria (ability to input international characters, display date formats, translate the user interface, and so on…). It is an efficient way to track progress made by each team over time and can even create some healthy competition among product teams. These teams are motivated to be on top of the i18n charts!
Rule 3 – “Integrate Localization into the Development Process”
To release a new product, development teams have many high-priority tasks and usually prefer not to have to worry about localization until necessary. As a consequence, localizability issues are often discovered too late and encounter the risk of being deferred to a future release. Product teams don’t always anticipate the impact of a particular task/feature on localization, and quite often, the Localization team isn’t able to influence design or development until the feature is already implemented.
In an agile process, features and development tasks are tracked in a backlog and reviewed at the beginning of every sprint. To eliminate the side effects of the “throw-over-the-wall” model described above, it is critical to include Localization representatives during these sprint-planning meetings so more visibility and importance are given to the localization tasks. This also provides great educational value to all stakeholders who can then understand the impact of their decisions on the localization process. Localization or a proxy should also attend the daily sprint meetings to keep up with the development pace and decisions. By attending these meetings, Localization team members can be much more proactive and influent.
Adobe Revel and Photoshop.com are examples of teams that integrate localization into their development process. They also prioritize localization intensive features/tasks upfront – carving enough time for the Localization team to run its process and deliver high-quality localized releases.
In a recent Localization World event, Amrit Singh (International Program Manager for our Installation technologies) presented LocBan (Kanban applied to Localization). Just like in a Toyota factory, the Localization team maintains a board of “To Do”, “Work In Progress” and “Done” tasks which provides great visibility on the localization “conveyor belt”. Similarly, it would be beneficial to maintain Kanban boards for each translator. In the waterfall model, translators used to receive large localization kits, which they had to scramble to complete within the deadline. In the agile world, translators are now able to “pull” work as their bandwidth opens up.
By using an integrated Kanban board, everyone has a clear understanding of all the various dependencies and accountabilities, resulting in stronger collaboration and higher success rate.
Rule 4 – “Reduce, Reuse and Recycle”
Localization can generate a lot of waste if not planned properly. So, it is key to become “green” in order to become more “agile”.
It is clear that reducing the localization effort will have a positive impact on a team’s agility. This could be achieved in 2 ways: by validating the localization scope and by reducing the translation waste generated during the localization process.
Reducing Localization Scope
The Localization Manager’s job is to ensure the company localizes the right product and content into the right language set. At Adobe, we have had situations in which we were localizing too much content. For example, using Adobe’s Digital Marketing Suite, we discovered that Russian customers prefer reading Development documentation (such as API descriptions) in English rather than in Russian. We were able to save a lot of time and cost by removing this component from our localization requirements.
Similarly, through market research, we discovered that most Middle-Eastern Creative Suite customers prefer to use an English user interface with Arabic or Hebrew documentation. This combination makes English content such as videos and tutorials more accessible to them.
In short, tracking web analytics and engaging with customers, power users, pre-release testers and geos constitute a great way to validate the localization requirements and improve agility.
Reducing Localization Waste
Once the localization requirements are confirmed, it is key to limit the translation waste generated during the localization process. This obviously impacts the translators’ work but also the bandwidth of the localization staff.
An effective way to reduce localization waste is by understanding its root cause. At Adobe, we categorize all localization defects through a common set of keywords, which provides us with a good picture of the issues faced across products. We can then develop solutions to reduce, if not eliminate, these defects.
Localization waste sometimes originates from English strings -assuming English is the source language. Indeed, translations created before English strings get finalized will need to be revisited and will likely generate some waste.
In the agile world, we can’t afford that extra time, so it is important to validate the English content before handing it off to the translators. Doing something as simple as spell checking can help to reduce a lot of localization waste. In a product such as InDesign, about 3% of the English user interface strings are updated once they get reviewed for spelling and grammatical mistakes. For a product that is localized into 25 languages, this represents a waste equivalent to 75% of a single language scope!
Also, many of the software localization testing activities are necessary because localization is happening out of context. Solving that problem can tremendously speed up the localization process. In an ideal world, localization should be a product feature that allows translators to translate the user interface in-context. Facebook did a great job in this area by enabling translators (in this case its user community) to translate and provide feedback within the application itself. Alternatively, translators should be provided context information through builds, screenshots or meta-data information (e.g. developer comments, feature name, expected delivery time, etc.).
To reduce waste, it is also recommended that localizers develop glossaries, style guides and tools that leverage previous localizations.
Ultimately, it’s critical for translators to validate their work as they translate. That way, activities down the production line can be eliminated or reduced, which makes the entire process more agile.
Reusing strings can sometimes be a source of challenging defects in software localization, so it has to be handled carefully. For example, the English string “none” could be translated as “aucun” or “aucune” in French based on the gender of the noun to which it refers.
That said, reusing strings – in the same context – could also help to improve agility, since these strings won’t need to be translated multiple times.
An area where Adobe has experienced positive results with reducing and reusing English content is in our instructional content. In documentation, Adobe relies on Acrolinx to control the quality of the English (source) content. Authors need to use a certain authoring style (e.g. shorter sentences) and are encouraged to leverage existing paragraphs (e.g. legal disclaimers). This improves consistency in the English documentation and has the great benefit of reducing the localization workload too.
Recycling is the process of transforming existing materials (or waste) such that they could be reused again – sometimes for a totally different purpose. Creating polar fleeces from used plastic bottles or isolating walls using old denim jeans are classic examples of recycling.
Such transformations can apply to translations too. Translators don’t need to translate every sentence from scratch. Translation technologies such as Translation Memories and Machine Translation engines can help translators recycle previous translations and speed up the translation process. At Adobe, we have experienced dramatic productivity gains when we used these technologies. In general, a translator supported by these technologies will deliver in an hour what other translators would deliver in a day. These are impressive gains that contribute to localization agility too.
Rule 5 – Automate, Automate, Automate
The last requirement to achieve agile localization is automation. With agile, you can’t afford to send translation requests through e-mails or cut and paste strings from a spreadsheet to a source file. All translation hand-offs should be automated and managed through a centralized system. Over the years, Adobe’s Globalization team has developed such a platform. This system is able to connect with various source control systems, manage translation jobs, leverage existing translations across projects and content types and provide machine translation engines. In the Globalization Myth 4 article, Guta Ribeiro introduced Airport, our new system to automatically connect with our vendors and help us march towards our lofty one-hour translation goal.
Beyond translation, it’s also important to automate other aspects of the localization process, such as build, quality assurance, bug fixing, screenshots and distribution of the localized releases. We can only go as fast as our slowest component, which is why it’s critical to automate all aspects of the localization process.
Localization agility can be achieved as long as all stakeholders work as a unified team. It is critical for core engineers to develop well-internationalized code from the get-go. This can be achieved via training, code reviews, usage of (Open Source) internationalization libraries and globalization report cards.
The localization process should be fully integrated within the overall development process so that all dependencies and accountabilities are clear. We recommend using Kanban boards (via tools such as Trello) to raise this visibility. To become agile, it’s also important to act “green” (i.e. reduce, reuse and recycle). Doing so represents an effective way to control waste generation before and during the localization process. Finally, all efforts should be made to automate all parts of the streamlined localization process.
At Adobe, not all localization projects are handled with great agility – yet! Some projects are more agile than others. However, based on our experience, we believe agility can be achieved by adopting these Five Golden Rules:
“We are One Team!”
“Internationalization is King”
“Integrate Localization into the Development Process”
“Reduce, Reuse and Recycle”
“Automate, Automate, Automate”
Special thanks to Rob Jaworski, Amrit Pal Singh, Ashish Saxena, Janice Campbell, Leandro Reis, Peter Green, Julia Feng and Quynn Le for their invaluable feedback on this article.
Adobe has a long history of developing products for multiple platforms, be it desktop applications like our flagship Creative Suite applications or newer touch applications like Photoshop Touch. Most of our desktop apps have been built for both Windows and Mac and newer applications continue on this trend with support for iOS and Android including Tablet and Phone form factors for both.
Of course this would not have been possible without the careful efforts of the engineering team to largely maintain a single code base for all platforms.
While having a single code base has obvious benefits, in the UI layer it is often important to have platform specific variations for better usability. Each platform usually has a specific convention for referring to system menus, short cut keys and UI elements. For example on a windows platform a UI String could be – “Select a media file via the Browse button or enter a valid pathname.” and the same string for the Mac Platform could be – “Select a media file via the Choose button or enter a valid pathname.”
This means that translatable UI strings may have many variations in the source language depending upon which platform they are intended for. This is what our globalization group usually refers to as ‘Platform Variance’. Localizable strings are essentially multivalued entities. Each localizable string has an identifier and multiple associated values each of which can be selected based on certain criteria. The most obvious and commonly used criteria is the UI locale of the application but it need not be the only one. Platform too can decide the value of a string.
Platform variance support is not just useful for handling terminology differences for referring to system UI elements, it also helps adapt strings for different screen sizes. Modern application are designed for supporting multiple device form factors like tablet and phone with the UI being tweaked for each platform for best user experience. Platform variance in this case can be used to support longer strings for the Tablet view and shorter strings for the Phone view.
Yet another area where platform variance support could potentially be useful is in having different localizable values for a Pro version versus a Consumer version of the application.
However, localizing strings with platform variant data is a problem. The problem is two fold, one is in managing the processes and project schedule to allow for agile localization and simultaneous release for all target platforms. The second aspect is technically supporting the platform variance in both programming libraries and translation tools. Many tools and libraries assume a single value for a source and a target string, but in case of platform variance not only can there be multiple source and target values for a string there need not be a one-to-one correspondence between source and target values. There may be multiple platform variants for a source string that map to the same translated/target value or a single source string may need to be translated differently based on platform for the target locale. For example:
en_US: “Please close the dialog and start over.”
default fr_FR: “Fermez la zone de dialogue et recommencez.”
Windows fr_FR: “Fermez la boîte de dialogue et recommencez.”
Since I am part of the globalization tools team here at Adobe, the remainder of this post I describe the problem more from a technical tools and libraries perspective, drawing from my experience. The process problem is also pretty complex and would probably take a much longer blog post to discuss. In fact there’s a related one already on this blog, see – link.
Platform Variance Support in Libraries
Ideally the globalization libraries/APIs used in the code to manage externalized strings and the corresponding storage formats for the externalized data should have a notion of a platform variant value for each string. There should be a way to request a string value for a specific locale and platform along with a provision to fall back to a default value in case a platform specific value is not specified.
As an example, the Java ResourceBundle API supports selecting a bundle by ‘Locale’, there is no explicit mention of a ‘Platform’, but the ‘Locale’ itself is extensible to support variants. The variant mechanism in the ‘Locale’ can be used for supporting different platforms and there is also a fall back mechanism. At Adobe we have a custom developed cross platform library called ZString for managing externalized strings with explicit support for platform variance.
Platform Variance Support in Translation Tools
Most translation management systems (TMSs) have a one-to-one model of source strings with matching translated strings for each locale. This assumption is behind the architecture of the TM matching algorithms as well as the design of the translation workbench. A typical translation workbench usually offers a side by side view of source and target strings, but only supporting a single source string corresponding to a single translated value.
We are still searching for the ideal solution to this problem. For managing the TMs a possible workaround using existing systems is to have duplicate entries in the Translation Memory (TM) or a separate TM for each platform.
However, translators are still constrained by the view presented by their translation workbench. A possible solution to allow translation vendors to provide platform specific translations is to duplicate all the source strings for each possible target platform. The source value for the default platform can be used as the source value for all other platform unless the application UI already specifies a value for a specific platform in which case that is used. Now the translator can provide different translations for each platform if required. This workaround however seems to be a significant amount of additional work for the translators. Some optimization is possible by translating a single platform first and leveraging translations for all the other platforms.
In an ideal scenario the translation workbench would provide a side by side view of all platform variants for the source string and the target strings. With the ability for the translator to remove variants from the translated string where they are not required and propose variants for the translated string even if the source string does not have any. This would allow translators to work through the source content in a single pass, editing leveraged translations, providing new translations where required and proposing platform specific translated values as appropriate.
An approximation to this ideal view is an Excel sheet with each source string being represented in a row and having a separate column for each platform for both source and target strings. With blank values in a platform column signifying that the default translation is to be used for that platform and non-blank platform entries being used for the platform specific translations.
We are still experimenting to find the optimal solution for our needs, that offers flexibility to translators and yet leverages our investment in existing translation tools and processes. The goal is to be able to support faster agile release cycles with all platform releases happening simultaneously.
I think this is a good forum to ask our blog readers if they have faced similar problems and the solutions they have developed to deal with it.
I have been asked lately to talk to a couple of peers in the industry about Marketing Localization at Adobe and thought this would make an interesting blog post as well.
At Adobe, Marketing Localization is centralized and consists of a team of International Program Managers, which I manage.
I believe this is still the right model for us. Adobe has offices all over the world and decentralizing marketing localization would actually introduce inefficiencies. That said, the challenge with a centralized model is to balance productivity with the ability to provide GEOs with the right process and tools so they can participate and provide valuable input around GEO-specific nuances, country specific content, etc. This is an on-going challenge and we work very closely with our Marketing Managers worldwide to conquer it.
The challenge here is the balance between giving more flexibility and freedom of expression to the regions and the use of productivity tools such as Translation Memory. If we want to leverage the savings that TMs and other tools offer to localization (and we do), we can offer some flexibility in the target content but not as much as sometimes the regions would like to have – for instance, complete re-writes of segments.
At Adobe we are aware to these issues and the key here is to work closely with the regional offices and offer them opportunities to provide feedback early on, directly into the source content, before localization starts. It also means providing opportunity for reviews on localized content that is presented in context in a process that allows for easy feedback. Our GEOs are Field Marketing Managers and we are sensitive to the amount of time spent in reviews.
There are always challenges in localization and in particular in Marketing localization, where ‘good translation’ is just not good enough.
Take the Digital Marketing BU for example. Worldwide campaigns around the digital marketing solutions have to appeal to ‘marketers’, to professionals that create marketing content, and so the ‘localization quality bar’ for the content we provide to our GEOs has been raised significantly.
Here’s an example of a recent campaign that was particularly challenging for localization due to the use of the very US centric expression “ticks them off”. For translators is not always a clear choice of words for the target language. The ‘weight’ of the expression and what it conveys need to be taken into consideration.
Regional offices are free to create their own marketing materials – International Brand Guidelines are in place and Adobe’s Brand team works directly with the GEOs to ensure a consistent interpretation and use of our brand.
We are very protective when it comes to the Adobe brand and although the regional offices are given some flexibility in terms of creating some of their own marketing materials (in their original language), Adobe’s Brand team normally is involved to make sure the materials follow the established international brand guidelines.
What regions or international markets are most difficult or challenging for localization?
I think anyone that works in localization would start by saying that Japanese is a very challenging language to localize. The idea of ‘translation’ is already something that is not appreciated by the Japanese market. A very good ‘translation’ still means ‘it’s translated’ and so it has a different flow and feel than the content that is originated in Japanese. This becomes an even greater challenge around marketing content.
At Adobe we do have a successful program for localizing marketing campaigns into Japanese and that involves working very closely with our in-country product marketing managers and employing in-country copy editors when necessary.
That said, every language and every country present challenges. We recently had to deal with orthography changes in Brazilian Portuguese, tonality changes in Spanish (formal to a more informal tone), imagery issues in the middle east, different ‘flavors’ of a single language, and so on. I guess I would say that there are no ‘easy’ regions. Our work is always interesting and we look at these challenges with a very positive attitude – these challenges are what differentiate translation from localization and it’s always exciting to be part of this process, where you see the source messaging deployed all over the world and having the intended appropriate impact in every region, around the globe.
What aspects of branding are most important to localize for regional audiences? What channels are most important?
The emphasis has greatly shifted to online content (web pages, multimedia content and social), the larger part of the content we now localize will end up on Adobe’s 57 international sites. The need for printed content has decreased but there are still certain regions that need to be supplied with printed content. We try to listen to our GEOs and provide relevant localized materials.
The Adobe ‘look and feel’ is very homogeneous in all our sites. Our regional offices have the flexibility to add country specific content but the site template is the same for all locales and all international sites are centrally managed.
Regional offices are also free to create some of their own marketing materials – International Brand Guidelines are in place and Adobe’s Brand team works directly with the GEOs to ensure a consistent interpretation and use of our brand.
I believe the big challenge now is the creation of a strong and successful Creative Cloud brand worldwide – and we are well underway 🙂
The regional offices should be an extension of your team and taken into consideration in every step of your processes and tools.
Creating global-ready, internationalized applications requires many people: engineers, project managers, translators, and often in-country experts. If everything goes as planned, the final product is internationalized and localized to meet the needs of a specific market. The required teamwork is amazing, and sometimes the expense can be surprisingly large. The mistaken conclusion is that internationalization must always be expensive, and that the effort simply costs too much in terms of schedules, resources, and of course money. The worst part about the conclusion is that the expense can be minimized considerably by rethinking when and how internationalization is performed.
Two causes of expensive internationalization are the delays in actively thinking about it and thinking of it as a simple feature. Often product managers and engineering teams simply do not plan for localization from the beginning of their project’s lifecycle. This is common in product teams that target English-only regions first. Unfortunately, engineers and product managers mistakenly think that they will be able to add the internationalization and localization “feature” at a later time when needed. This is an expensive mistake, and it comes from thinking of internationalization as a feature instead of an architectural and design style. The result is that the final product does not contain any framework for localization and has not been designed with internationalization and localization in mind. Ultimately, it is difficult and expensive to retrofit or “fix” an application that has a single language architecture and design. Internationalization simply touches too many areas of a product to be considered as a one-time feature that can be added to the product sometime in the future.
You can save yourself the expense and difficultly of retrofitting or fixing an English-only product by thinking of the internationalization step as an architectural element rather than a feature. A feature can be readily added to a product often because it has limited scope within the application or has few dependencies. A new feature is often “low-touch” or only lightly coupled with other features or areas of a product. Internationalization, however, often affects all aspects of an application. Areas of the product that involve number, date, time and currency formatting can cut across many areas. Internationalization is a “high-touch” activity that affects most areas of an application because user interfaces, strings, icons and colors, numbers, dates, and time values are used throughout an application. Finding and fixing those areas so that they are internationalized and localizable is an onerous task once the product already exists and is in production.
If you architect your product from the beginning so that localizable elements are isolated from core business logic, you make the localization task easier later. How can you think about internationalization as an architectural task rather than as a feature?
First, understand that internationalization will affect many aspects of your system. Think about all the areas that utilize strings and other localizable resources. The list may be bigger than you first imagined.
Secondly, after identifying those areas, extract those text strings and other resources so that they can be translated independently without touching your application’s source code. Every programming platform has a means of isolating resources from the core application. Learn about that mechanism and use it. Think of this step as creating the generic, language-neutral scaffolding upon which the rest of your application will be built. You want to create a core set of business logic and user interface layout that is independent from language and culture. Later, the language-specific elements can be translated and added onto this core architecture.
Lastly, architect your application to load needed language modules at runtime rather than having them hard-coded into the application. Placing the right internationalization architecture in the product from the beginning costs little in terms of time lines and resources, and it pays off significantly over time when product teams discover that their “English-only” application is now desired in new language markets.
This article was originally written in English. Text in other languages is provided via machine translation.
Adobe’s Globalization team is committed to driving continuous improvement in customer experience and improving efficiency. These are two of Adobe’s areas of focus for 2013.
The top customer feedback we used to receive was that localization of our products needed to be more agile. The digital video team, for instance, moved to releasing English and localized products in one installer; they needed to sim-GM in order to release quickly and meet the market expectations. The creative products customers in Brazil and Russia, for instance, were getting tired of waiting six months to get their hands on a localized product. Like many in the industry, Adobe’s Globalization team began to face pressure to localize with fewer resources, reduce localization turnaround time, and improve quality.
Our challenge is to meet the aggressive release schedules of Adobe’s Clouds—Digital Media and Digital Marketing, their companion products and tools in 20+ languages. Currently, our team uses, roughly, 65% of our resources on Digital Media localizations, and 15% on Digital Marketing localizations (20% is dedicated to Print, tools development, finance, and other initiatives). The localization team is made up of approximately 150 people—international program managers, international engineers, international quality engineers, and interns located in the US, China, India, Japan, and Romania, supporting approximately 150 product and functional teams.
In this paper we will show how Adobe has been able to accelerate localization and we will, hopefully, debunk the belief that localization takes 6 weeks. With limited budget, we are meeting expectations, sim-shipping English and 20+ languages for the Adobe Cloud-based products—Digital Media and Digital Marketing– Developer/Web tools, and Touch Apps. We support any agile workflow and release schedules can be as short as every 2-4 weeks (Photoshop.com, Acrobat.com, Cloud Manager, adoberevel.com), 6 weeks (DPS, AdobeRevel), to continuous releases (CCM) varying from twice a week to monthly. In these workflows, we can complete a localization cycle as quickly as within 24 hours. Our ultimate goal? We are preparing for the day when Adobe products get released multiple times a day!
Sample SCRUM Schedule – Localized product sim-releases in 20 languages every 6 weeks
Slow is History! – Product Team Concerns in the Past and Globalization Team Response
Adobe product development models come in many flavors. In general, the ‘waterfall’ model was considered standard. Products such as Photoshop, Acrobat, InDesign, and Illustrator were well-suited for standard localization. That is, at UI Freeze milestone, the localization team would step in and begin the localization process. This meant that, by English GM, the localized versions were lagging behind by 4-6 weeks.
This localization model—start localization after UI Freeze milestone—had several drawbacks. Localization partners got overwhelmed at end game; localization issues were found too late and, when issues were classified as “show-stoppers,” they could jeopardize the product release schedule; many critical defects ended up getting deferred for the next product release. The model was costly, time consuming, it increased stress and burnout.
With the event of multi-lingual installers, SaaS, Cloud-based products, and new development workflows used by Adobe product teams such as Scrum, Kanban, Scrumban, or Adobe’s own Aphid, the Globalization team was faced with new requirements which called for a more accelerated localization workflow. The new business model required that localization should be possible in any locale, be scalable to a large set of languages, and respect various budget constraints. Product teams started to question whether the localization team was ready to keep up with the pace of the business. The answer was a big “YES, OF COURSE!”
For the sake of comparing two different models, see below a waterfall-model localization schedule and an agile localization schedule from the same period—2004. The product following the waterfall model on the left, chose to ship single-language versions in a staggered schedule; the one on the right, based on Adobe’s Aphid model, released their product with a multi-lingual installer as a single binary. At this point in time, Globalization was ready to accommodate both Product Teams.
Waterfall – InDesign Agile – Audition
With the acquisition of Macromedia in 2005, Adobe Localization faced many new and exciting challenges. Dreamweaver, for instance, ran prerelease programs for all languages, which meant that localization had to be right behind English, at the same level of development, testing, and stability as English. Thanks to the many talents that we acquired and the level of cooperation between companies, we were able to leverage best practices and meet the expectations of the new Product Teams.
In 2007, Adobe acquired Scene7, a software company enabling websites to have real-time rich media; then Omniture, in 2009. In 2010, Adobe acquired Day Software, a market leader in next-generation Web Content Management. In 2011, web tool companies such as TypeKit and Nitobi, the maker of PhoneGap; Efficient Frontier, a digital marketing company; and Iridas, the maker of SpeedGrade, a professional color correction application; in 2012, Adobe acquired Behance, the leading online platform to showcase and discover creative work.
All these new cloud-based services demanded improved localization velocity, mostly through internationalization and automation. Development cycles were short – 2-6 weeks – and updates were frequent and web based. Our team needed to shift focus in order to meet these cloud-based product requirements.
The Globalization team, in partnership with the Product Teams, worked towards an agile localization model. Here were the first steps we took towards acceleration:
Ensured Adobe products were World Ready — created an assessment tool called the Globalization Report Card (GRC) to determine world readiness compliance.
Engaged international QE early in the development cycle to test and report internationalization bugs. With that, international bugs were addressed in a timely manner which helped speed up the development cycle and improve products’ quality. See “Globalization Myth Series – Myth 2: This software product is only for the U.S.” for evidence on the benefits of addressing internationalization issues at the beginning of the development cycle.
Worked cooperatively with the new Development Teams to set common goals and share best practices. This cooperation has resulted in better success rates than if localization were considered as an after-thought and as a different team.
Changed the mindset – if we got content EARLY and ITERATIVELY, we would localize software and documentation continuously. Waterfall-based products started working with the globalization team prior to UI Freeze milestone. We started localizing glossary kits early and testing localized builds sooner than in the past. Documentation team started handing off non-final files prior to the usual “screen-shot ready build” milestone. Localized documentation is now uploaded to the web at the same time as the English product and localization versions ship.
Developed more tools – invested in ALF (the Adobe Localization Framework is a multi-tiered system whose primary intent is to automate the localization process and facilitate the creation of localized products), machine translation (takes strings in one human language and automatically translates them into other human languages), World Server (an enterprise translation and globalization management system that enables Adobe to simplify and accelerate our translation and localization processes for any content, from our websites to instructional content to software applications and beyond), tools that helped achieve localization of fast releases nearly in sync with English.
Engaged our external localization partners earlier and began growing expertise in those teams. Some of our vendors now are capable of offering turnkey localization services for Adobe products. They have learned our products through prerelease, product demos, and training by our Globalization team.
Found new ways to engage with our customers through the Adobe Translation Center, international prerelease, and forums. These customers know our products best, so they can provide early feedback and we can save time in the long run.
With these measures in place, we were able to get closer to the English product schedule and were behind by just a couple of weeks.
Currently, the Globalization team has reorganized internally to manage the localization process for the end-to-end customer experience which includes software, marketing/web content, documentation, internationalization/localization testing, and educational materials. We are better aligned and more agile, able to support localization for product cycles of 2-4 weeks (Photoshop.com, Acrobat.com, Cloud Manager, adoberevel.com), 6 weeks (DPS, Adobe Revel), to continuous releases (CCM) varying from twice a week to monthly. In order to achieve such challenging schedules, we are turning localizations around in 24 hours at times.
For instance, Adobe Creative Cloud is released with 16 localized components and its schedule today looks like this:
Standard, large projects schedules like Flash CS6, look like this:
Looking Forward — Introducing A.L.A., the Adobe Airport
One of Globalization’s biggest challenges is to meet the aggressive release schedule of Adobe’s Clouds, its companion products, web tools, and touch apps in 20+ languages.
In the same way that planes have to leave on scheduled time, crew and passengers in place, in a continuous flow, so do our localized products, where the crew is formed by International Program Managers, International Quality Engineers, International Engineers and ‘passengers’ are the assets (strings) for translation, from any number of products. The plane may only be going to France (say) or it may be doing stops in numerous countries. The frequency of flights will also vary, depending on demand.
We started out with pilot projects, mostly projects which needed quick localizations at end game. Our requirements were:
Source strings were final and reviewed (proofread)
Our Product Teams’ expectations were:
Human translations compliant with Adobe terminology
No functional/linguistic testing
Note that even though our tool collects strings from multiple products and sends them to the translators, we can guarantee terminology consistency and quality because the strings are first leveraged through World Server; and our translators make use of our glossaries for reference.
The Adobe Localization Airport (A.L.A.) aims to provide one-hour localization turnaround. Our tool collects strings from multiple products and sends them to the translators. Once translated, the strings are distributed back to their respective products, presto!
Turn-Around-Time (TAT) – from Q2 2012 to Q1 2013, TAT has decreased from an average of 12.7 hours to 7.8 hours. Our goal, by end of Q2 2013, is to reach 1 hour TAT or less, should the project require that much acceleration.
Airport Turn-around-time per language, per quarter
Localization does not take 6 weeks—it takes an average of 7.8 hours. Our goal, by the end of Q2 2013, is to reach 1 hour TAT or less! This is a myth that we have debunked. The Globalization tools and initiatives aim to make localization at Adobe even more agile, without compromising our products’ quality as well as customer satisfaction.
The A.L.A. team—Ajay Kumar, Guta Ribeiro, Joel Sahleen, John Nguyen, and Warren Peet
Quynn Megan Le