Skip to content
View nis12ram's full-sized avatar
🏠
Working from home
🏠
Working from home
8 followers · 33 following

Achievements

Achievement: Pull Sharkx2

Achievements

Achievement: Pull Sharkx2

Block or report nis12ram

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
nis12ram/README.md

Hi there, I'm nishant! 👋

🔭 Currently working on Lakehouse Catalog and Governance, Query Engines, and Cloud
🌱 Learning Rust

🌐 Socials:

LinkedIn

💻 Tech Stack

Rust Python SQL Apache Spark Apache DataFusion Delta Lake Unity Catalog Azure Databricks Azure Git GitHub VS Code

📊 GitHub Stats

GitHub Streak

🌱 Open Source

sail arco

🚀 Cloud Data Pipeline Projects

Multi-Region Sales Data Pipeline (UnifiedSalesReportingPipeline)

Designed and implemented an end-to-end Azure lakehouse pipeline to consolidate ~12M daily sales records from each regional branch into a unified dataset.

  • Orchestration: Built modular, fault-tolerant workflows in ADF with centralized logging, retry logic, and restart-from-failure handling.
  • Transformation: Developed parameterized notebooks in Databricks for cleaning, normalization, and metadata enrichment.
  • Governance: Implemented data quality checks by quarantining invalid records and enforcing NOT NULL/CHECK constraints.
  • Storage: Wrote optimized, ACID-compliant Delta Lake tables on ADLS Gen.

Github Link

UnifiedSalesPipeline
Agricultural Commodity Price Arbitrage Pipeline (agri-price-arbitrage-adf-pipeline)

Built an ADF-driven pipeline to ingest and process daily agricultural commodity price data from external APIs.

  • Architecture: Utilized a Medallion (Bronze-Silver-Gold) architecture for structured data processing.
  • Data Flow: Cleaned and standardized nested JSON data using ADF Data Flows to ensure consistent schema and quality.
  • Analytics: Implemented business logic to identify in-state price arbitrage opportunities for actionable trading insights.

Github Link

agri_poster1

I enjoy documenting my Data Engineering learnings in simple, digestible explanations to improve my understanding. Posts Website: DEDigest

Pinned Loading

  1. sail sail Public

    Forked from lakehq/sail

    Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

    Rust

  2. arco arco Public

    Forked from daxis-io/arco

    Serverless lakehouse infrastructure - file-native catalog and execution-first orchestration

    Rust

  3. UnifiedSalesReportingPipeline UnifiedSalesReportingPipeline Public

    UnifiedSalesReporting is an Azure-based data engineering solution designed to automate the ingestion, transformation, and reporting of daily sales data from multiple regional branches

    Jupyter Notebook

  4. HindiNER-v0 HindiNER-v0 Public

    Code for HindiNER-v0 - a general purpose Hindi NER model

    Jupyter Notebook

  5. sqlite3-vector-db sqlite3-vector-db Public

    a simple vector database built on top of sqlite3.

    Jupyter Notebook

  6. in-browser-vector-db in-browser-vector-db Public

    in-browser vector database built using javascript

    JavaScript