GSoC 2021 — Improving Apache OODT OPSUI React.js UI with Advanced Functionalities
My experience with Apache OODT as a GSoC 2021 student
--
Google Summer of Code (GSoC) is a global program hosted by Google that is focused on bringing more student developers into open source software development. Students work with an open-source organization on a 10-week programming project during their summer break from school.
Through GSoC, Open source projects get their work done and attract more developers into their open-source endeavors while the students get experience contributing to open source and learning new things, also getting paid for their effort.
I was privileged to participate in the GSoC this year, for the Apache OODT project of the Apache Software Foundation. This article describes my GSoC 2021 project, Improving OPSUI React.js UI with Advanced Functionalities (OODT-1035). Through this post, I will give an introduction to my project, how I came up with the implementation, challenges faced and the experience I gained, along with the possible future developments/improvements to the project.
Apache OODT In General
Apache Object-Oriented Data Technology (OODT) is a software framework that lets users integrate and archive their processes, data, and files’ metadata. It provides services for data-generating, processing, managing, and distributing. Further OODT allows its users to analyze their data while allowing for the integration of data, computation, visualization, and other components.
It is created by NASA as a middle-ware framework for sharing data across heterogeneous, distributed data repositories in 1998.
Apache OODT can be recognized as,
- A middleware for Metadata — Distributed processing and virtual archives, migrate access to distributed resources, data discovery, and query optimizations
- A Software Architecture — Information representation model, unification of data, technology, and metadata.
Apache OODT is a combination of 3 major sub-components.
- File Manager — Data Integration
- Workflow Manager — Executing workflows
- Resource Manager — Managing and scheduling jobs
My Project: [OODT — 1035] Improving OPSUI React.js UI with Advanced Functionalities
Overview
At the present stage, Apache OODT provides a web app to monitor the status of each component and ingested files, metadata, and workflows. This main dashboard is known as OPS UI which is based on the Apache wicket java web framework.
Though it provides basic monitoring functionalities like retrieval of product status, metadata, workflow, and platform health information, etc. it lacked some important features like querying over products, product removal, and workflow termination, etc.
To address all those loopholes, a new component-based React UI with enhanced REST APIs had been started to develop in mid-2019. It had been planned to release with Apache OODT 2.0. However, some of its functionalities were yet to be completed by 2021.
Therefore, my project aims to implement the missing features of React. js-based OPSUI and make it ready to be released with the Apache OODT 2.0.
Objective
The objective of this project is to complete React. js-based OPSUI by implementing the following features.
- Implement querying functionality at OPSUI side (scope can be determined)
- Show progress of workflows and file ingestions
- Introduce a proper REST API for the resource manager component
- Introduce proper packaging (with configurable external REST API URLs) and deployment mechanism (as a docker deployment )
At the time of writing this, all goals except implementing the REST API for the resource manager component have been completed.
Deliverables
Following are the outcomes which are delivered at the end of the project period.
1. Improved UI/UX in the product ingest and product browser pages with proper error handling.
2. Improved UI/UX in the workflow browser page with proper error handling
3. Medium article on Setting up Apache OODT development environment with IntelliJ IDEA
Design and Implementation
Work completed
- [OODT-1035][OODT-1037] Improve UI/UX of Workflow Manager Components in React.js OPSUI https://github.com/apache/oodt/pull/128
- [OODT-1035]Improve UI/UX of File Manager Components in React.js OPSUI https://github.com/apache/oodt/pull/126
- [OODT-1035] Improve product browser and product ingest UI https://github.com/apache/oodt/pull/123
- [OODT-1035]Refactor React OPSUI sample app https://github.com/apache/oodt/pull/121
Challenges faced
The main challenge I faced during the GSoC was the unexpected academic activities and family problems. Though it was initially appeared to be commitment-free, due to the pandemic-related problems, the lecture and exam schedules of my university were unexpectedly changed. Therefore I had to work hard to balance GSoC work and academic activities.
In addition, I was somewhat novice to Java development in a larger project like Apache OODT. Therefore I had to learn a lot of things including setting up the development environment, the Jersy framework, and the maven ecosystem. Since much of my time was spent on learning and getting familiar with the tech stack, I couldn’t complete the implementation of the REST API for the resource manager component.
Future Development
- Hosting the new OPSUI on the Apache Infrastructure.
It had been planned to host the new React.js OPSUI on Apache infrastructure so that the new users will be able to test the product without messing with installing and configuring the OODT components. However, it couldn’t be done within the project period due to the lack of time and now it’s temporarily hosted on my Heroku account. I intend to continue working on this after the GSoC is completed.
- Continue the development of the REST API for the resource manager component
Currently, the React.js OPSUI doesn’t enable users to view resource manager-related data (such as job queues, job status, etc). Therefore it had been planned to implement a REST API for the resource manager component as a part of this project. However, I was not able to complete it within the GSoC time period. Therefore, I will be working on that even after the GSoC is finished.
Thank You All!
“If I have seen further than others, it is by standing upon the shoulders of giants.” -Sir Isaac Newton
This was the first time I participated in GSoC. I had been contributing to the Open Source before GSoC, but they were mostly documentation and translation work. This was the first time that I was able to make such a big contribution in terms of programming/development. This was all possible due to the continuous support and guidance I received from my mentors.
Imesha Sudasingha is the current project lead of the Apache OODT project and was one of the mentors for my project. He showed continuous support and helped me in every problem including setting up the development environment and patiently explaining the complexities of the project.
Nadeeshan Gimhana is a PMC member of the Apache OODT project and acted as a mentor for my project. In fact, he was the one who had initiated implementing the React.js OPSUI as his GSoC project in 2019. Therefore, he had a clear picture of what had already been done and what needed to be done. This helped me to rapidly progress through the project.
Chris Mattmaan was always there when we were stuck somewhere while in development. He had broad knowledge about the entire codebase, therefore any frustrations were no more there, thanks to him.
The most important thing I admire is the patience of my mentors. Though I expected this GSoC time period to be commitment-free, unexpected family problems and exams that appeared didn’t let me work at my full potential. Nevertheless, my mentors patiently continuously supported me to complete the project and I am really grateful to them for that.
Finally, I would like to thank the entire OODT team and the Apache Software Foundation as a whole for every piece of advice and support given through Slack and the mailing list. Without them all, I wouldn’t have been able to make this project a success.
Thank you, Google!
Without Google, I wouldn’t have got this opportunity to meet these skilled, passionate people and get this valuable experience by contributing to OODT. Thank you, Google! Keep it up! :-)
