EXPERIENCE / PLATFORM, PRODUCT, PRODUCTION

Infrastructure.
Real-world scale.

Infrastructure Platform Engineer IV at UF IT. Building the gateways, real-time services, and AI products people rely on.

9B+tokens / day

Shared AI platform behind the LiteLLM gateway I maintain and extend

2,000+concurrent streams

Real-time token delivery across 12+ generative AI workflows

10+microservices

Production services deployed and operated on Kubernetes

ENGINEERING / CONTROL PLANE TO CLIENT

The systems
behind the scale.

IDENTITY + ROUTING

Shared AI gateway

  1. Applications
  2. LiteLLM gateway
  3. Model providers

API-key authentication, RBAC and team isolation define access. Provider routing and failover, team budgets, and audit logging keep shared usage accountable.

LiteLLMAPI keys / RBACBudgetsAudit logging
9B+ tokens/day on the shared platform

WORKERS + EVENT DELIVERY

Concurrent token streaming

  1. Celery workers
  2. Redis Pub/Sub
  3. SSE clients

Background execution and event delivery are separate from the browser-facing stream. My work redesigned this path for concurrent token delivery across generative AI workflows.

FastAPICeleryRedis Pub/SubServer-Sent Events
2,000+ concurrent streams · 12+ workflows

EVIDENCE + REGRESSION CHECKS

Retrieval and evaluation

  1. OCR + chunks
  2. Hybrid retrieval + rerank
  3. Source-attributed answer

Document extraction and metadata-filtered retrieval feed answers with source attribution. Evaluation checks relevance, groundedness, tool calls, schema conformance, latency, and provider failures.

BM25EmbeddingspgvectorEvaluation harnesses
Excel/PDF analysis · PostgreSQL/pgvector

01 / May 2025 — Present

University of Florida

Information Technology

Infrastructure Platform Engineer IVGainesville, FloridaCURRENT ROLE

AI infrastructure at university scale.

I maintain and extend UFIT’s LiteLLM-based gateway on a shared platform processing 9B+ tokens per day, and build the products and services connected to it.

9B+ tokens / day on the shared platform

A gateway for shared AI access

Provider routing and failover, API-key authentication, RBAC, team isolation, budgets, and audit logging for multi-tenant LLM access.

  • Control plane: API keys, RBAC, team isolation, budgets, and audit records.
  • Provider path: request routing and failover across model providers.

Product + platform ownership

NaviGator Prompt Builder

Led development across Next.js, TypeScript, and FastAPI, connecting prompt creation, evaluation, and sharing to backend AI services.

  • Product layer: Next.js and TypeScript prompt-creation, evaluation, and sharing flows.
  • Service layer: FastAPI endpoints connected to backend AI services.

2,000+ concurrent streams

Real-time delivery

Redesigned streaming with Redis Pub/Sub, Celery, and Server-Sent Events for 2,000+ concurrent token streams across 12+ generative AI workflows.

  • Execution: Celery background workers and Redis Pub/Sub event delivery.
  • Client transport: Server-Sent Events for 2,000+ concurrent token streams across 12+ workflows.

Retrieval + evaluation

Grounded answers and evaluation

Built OCR, chunking, hybrid retrieval, reranking, and source-attributed document pipelines, with regression harnesses for groundedness, tool calls, schema compliance, and provider failures.

  • Ingestion: OCR fallback, chunking, embeddings, and metadata filters.
  • Retrieval: lexical + dense candidates, reranking, and source attribution.
  • Regression checks: relevance, groundedness, tool-call correctness, JSON schema, latency, and provider failures.

Data + reasoning

A departmental Data Science Agent

Combined TimesFM forecasting, BM25, and dense retrieval over PostgreSQL/pgvector to work with Excel and PDF datasets containing hundreds of thousands of rows.

  • Data sources: Excel and PDF datasets with hundreds of thousands of rows.
  • Query path: BM25 + dense retrieval over PostgreSQL/pgvector.
  • Analysis path: TimesFM forecasting alongside document and tabular workflows.

10+ production microservices

Operating the platform

Deploy and operate 10+ Kubernetes microservices with Terraform, F5 load balancing, GitLab CI/CD, health checks, automated tests, and staged rollouts.

  • Infrastructure: Kubernetes workloads, Terraform, and F5 load balancing.
  • Release path: GitLab CI/CD, automated tests, health checks, and staged rollouts.
LiteLLMKubernetesFastAPINext.jsRedispgvectorTerraformGitLab CI/CD

02 / Aug 2022 — Dec 2023

Tata Consultancy Services

Enterprise platforms

Systems EngineerHyderabad, India

Reliable services behind enterprise workflows.

Built Java Spring Boot and Node.js services supporting 100K+ daily user interactions, with stable APIs and SQL-backed workflows across business units.

100K+ daily interactions

APIs that teams can build on

Request validation, idempotent processing, and backward-compatible REST contracts for independently deployed frontends and downstream systems.

  • Service interfaces: Java Spring Boot and Node.js REST APIs.
  • Contract handling: validation, idempotent processing, and backward compatibility.

SQL + service optimization

Data performance

Tuned PostgreSQL and MSSQL query plans, joins, indexes, and connection usage, reducing average API response time by approximately 50%.

  • Database layer: PostgreSQL and MSSQL query plans, joins, indexes, and connection usage.
  • Reported result: approximately 50% lower average API response time.

Workers + CI/CD

Asynchronous work and repeatable releases

Moved high-volume ingestion into workers with retries and rate limits, and maintained GitLab CI/CD pipelines for testing, packaging, and multi-environment deployment.

  • Worker path: asynchronous ingestion with retries and rate limits.
  • Release path: GitLab CI/CD for tests, packaging, and environment deployment.
JavaSpring BootNode.jsPostgreSQLMSSQLGitLab CI/CD

EDUCATION

The academic foundation.

M.S. in Computer Science

University of Florida

Jan 2024 — Dec 2025

Gainesville, Florida

B.Tech in Computer Science

Jawaharlal Nehru Technological University Kakinada

Aug 2018 — Jun 2022

Kakinada, India
View the current resume