Ryan Walters 1c6d2d4b6e perf: implement batch operations and optimize database indexes
Add batch upsert functionality to reduce database round-trips from N to 1 when inserting courses. Create comprehensive database indexes for common query patterns including term/subject lookups, time-series metrics, and job scheduling. Remove redundant indexes and add monitoring guidance for BRIN index effectiveness.
2025-11-03 11:18:42 -06:00

banner

A complex multi-service system providing a Discord bot and browser-based interface to UTSA's course data.

Services

The application consists of three modular services that can be run independently or together:

  • Discord Bot (bot)

    • Primary interface for course monitoring and data queries
    • Built with Serenity and Poise frameworks for robust command handling
    • Uses slash commands with comprehensive error handling and logging
  • Web Server (web)

    • Axum-based server with Vite/React-based frontend
    • Embeds static assets at compile time with E-Tags & Cache-Control headers
  • Scraper (scraper)

    • Intelligent data collection system with priority-based queuing inside PostgreSQL via sqlx
    • Rate-limited scraping with burst handling to respect UTSA's systems
    • Handles course data updates, availability changes, and metadata synchronization

Quick Start

pnpm install -C web  # Install frontend dependencies
cargo build  # Build the backend

just dev # Runs auto-reloading dev build
just dev --services bot,web # Runs auto-reloading dev build, running only the bot and web services
just dev-build # Development build with release characteristics (frontend is embedded, non-auto-reloading)

just build # Production build that embeds assets

Documentation

Comprehensive documentation is available in the docs/ folder.

Description
No description provided
Readme 1.9 MiB
Languages
Rust 88.5%
TypeScript 8.4%
Dockerfile 1.1%
Just 0.8%
JavaScript 0.6%
Other 0.6%