Portfolio

Financing Process Automation for Sightseeing Tours & Riverboat Dining Cruise Company

Technology Used: Python, PostgreSQL, Microservices, ReactJS, RabbitMQ, Celery, Toast Integration, FareHarbor Events, Amadeus Data Integration, Stripe Events, Sage Accounting System Integration

Project Overview: This web application streamlines and automates the financial processes for a cruise company, integrating various point-of-sale systems and ensuring seamless data collection, verification, and transaction processing. The system is designed to improve operational efficiency and accuracy, reducing manual efforts and errors.

Key Features:

• Data Collection: Automates the collection of sales data from multiple sources, including FareHarbor bookings, Toast POS devices, and Stripe payment processing.
  

• Ticket and Payment Workflow: Manages the workflow for ticket sales, payment collection, and data verification.
  

• Data Reconciliation: Reconciles data based on tickets sold and collected information from third-party systems, ensuring data integrity and accuracy.
  

• Reporting: Generates comprehensive reports of reconciled data for business owners, providing valuable insights and facilitating business analysis.
  

• Dashboards: Offers user-friendly dashboards for real-time monitoring and management of financial data.
  

Technical Details:

• Data Integration:
  - Utilizes webhooks to collect data from FareHarbor.
  - Extracts data from Toast POS via CSV files stored at a designated location.
  - Incorporates Amadeus and Stripe events for comprehensive data gathering.
  

• Data Processing - Implements Celery jobs to process FareHarbor data and CSV files from Toast, storing the processed data in a local PostgreSQL database.
  

• Architecture - Employs a Domain-Driven Design (DDD) architecture for REST APIs, ensuring scalable and maintainable code.
  

• User Interface - Develops a ReactJS-based UI for users to view and verify different types of revenues and taxes.
  

• Verification Workflow - Designs a robust workflow for the verification of collected and reconciled data.
  

• Sage Integration - Creates and uploads CSV files to import transactions into the Sage Accounting System, completing the automation loop.
  

This project not only enhances the accuracy and efficiency of the cruise company's financial operations but also provides a robust platform for future scalability and integration with other business processes.


Performance Advisory(Energy Domain)

Project Overview: This performance advisory application was developed for a mechanical-based organization to monitor and analyze the performance of various devices. The application parses data extracted from different sensors, such as temperature, fuel level, and electric load, computes performance metrics for a given time frame, and displays these metrics on a dashboard. Additionally, the data is used by a machine learning team for future metric predictions and anomaly detection.

Technical Specifications:

• Data Ingestion: Sensors publish data to Kafka topics.
  

• Data Processing: Apache Airflow consumers extract and transform sensor data into the desired format.
  

• Storage: Processed data is stored in PostgreSQL.
  

• API Development: GraphQL endpoints are created to display performance metrics on the frontend.
  

• Role and Responsibilities:

  • Kafka Design: Designed Kafka topics, groups, and queues to handle sensor data efficiently.

  • Data Extraction: Created Apache Airflow DAGs to extract data from Kafka topics.

  • Data Transformation: Transformed raw sensor data into the required format and inserted it into PostgreSQL.

  • GraphQL API Development: Developed GraphQL queries and mutations to interact with the UI layer.

  • Testing: Wrote unit test cases using pytest and unittest to ensure code quality and reliability.

  • Issue Resolution: Addressed reported issues and performed optimizations to enhance application performance.

  • Deployment and CI/CD: Set up Nginx, Gunicorn, and CI/CD pipelines using Jenkins to automate deployment processes.
    

Technology Stack:

• Backend: Flask

• Database: PostgreSQL

• Design Patterns: Repository pattern

• Message Broker: Kafka

• Data Processing: Apache Airflow

• Cloud Services: AWS (EC2, CloudWatch, EKS, DynamoDB)

• Web Server: Nginx, Gunicorn

• CI/CD: Jenkins
  

Outcome: The application effectively monitors and analyzes device performance, providing valuable insights through a user-friendly dashboard. The integration with Kafka and Apache Airflow ensures real-time data processing, while the use of GraphQL allows for efficient data querying and interaction with the frontend. The system's reliability and performance are further enhanced through continuous testing and optimization efforts.


Designing and Implementing a Microservices for PDF Generation using Quadient Templates and Pipelines

Technologies: DAPR, FastAPI, XML Parsing, Azure Blob Storage, AWS S3, Quadient API Integration, MySQL, Flyway Database Versioning, Redis

Project Overview: This project focuses on designing and implementing a microservice capable of generating PDFs using Quadient templates and pipelines. The microservice accepts data requests in predefined JSON/XML formats, processes them through Quadient-compliant pipelines, and provides clients with the ability to download the generated PDFs. Additionally, it includes success and error callback mechanisms to ensure reliable communication and tracking.

Key Features:

• Asynchronous Processing: The service is fully asynchronous, efficiently handling PDF generation requests and providing non-blocking operations.

• Consistent API: Provides a consistent API for clients, abstracting the complexities of the underlying third-party Quadient service.

• Task Status Query: Enables clients to query the status of their submitted tasks, ensuring transparency and real-time updates.

• Error Resilience: Incorporates an error-resilient architecture with exponential backoff retry mechanisms to handle transient failures.

• Template Microservice: Utilizes a Cookiecutter template for designing and implementing the microservice, ensuring a standardized and modular codebase.

• Request Queue Management: Implements a request queue to handle request bursts and manage third-party processing limits effectively.

• Audit Logging: Records audit logs for each request, allowing administrators to analyze client behavior and ensure compliance.

• Data Transformation: Transforms input XML data to a predefined JSON format using XML to JSON mappings, facilitating seamless integration with Quadient pipelines.

• Callback Facility: Implements success and error callback mechanisms, with robust error handling and retry capabilities to ensure reliable communication.

Technical Details:

• Asynchronous Service Design: Implemented a fully asynchronous service to accept and process PDF generation requests, ensuring non-blocking operations and efficient resource utilization.

• API Abstraction: Developed a consistent API for clients, abstracting the complexities of the Quadient service and providing a simplified interface for PDF generation.

• Task Status Query: Enabled clients to query the status of their submitted tasks, providing real-time updates and ensuring transparency in the process.

• Error Resilience: Designed the microservice to be error-resilient, incorporating exponential backoff retry mechanisms to handle transient failures and ensure high availability.

• Template Microservice: Utilized Cookiecutter for designing and implementing the microservice, ensuring a modular and standardized codebase.

• Request Queue Management: Implemented a request queue to handle bursts of requests and manage processing limits imposed by third-party services.

• Audit Logging: Developed functionality to record audit logs for each request, providing administrators with valuable insights into client behavior and ensuring compliance.

• Data Transformation: Transformed input XML data to a predefined JSON format using XML to JSON mappings, ensuring compatibility with Quadient pipelines.

• Callback Mechanism: Implemented success and error callback mechanisms, with robust error handling and retry capabilities to ensure reliable communication and accurate status reporting.
  

This microservice not only automates and streamlines the PDF generation process but also provides a robust and scalable solution for integrating with Quadient templates and pipelines, ensuring high performance and reliability.