Accepted
FeedForward requires a database system to store user information, assignments, submitted drafts, and feedback. We needed to select an appropriate database technology considering:
Expected Usage Patterns: - Up to 20,000 total users at the institution level - Approximately 1,000 active users at any given time - Around 100 concurrent feedback requests - Predominantly read operations after data is written
Performance Requirements: - Non-real-time feedback generation (1-2 minute processing time is acceptable) - Data consistency is important for academic records - Local deployment within institutions
Development and Operational Constraints: - Limited resources for database administration - Preference for technologies compatible with the Python stack - Need for simplicity in deployment and maintenance
Alternatives Considered: - PocketBase: An open-source backend with built-in user management and SQLite storage - PostgreSQL: A full-featured client-server database - MongoDB: A document-oriented NoSQL database - Firebase/Firestore: Cloud-based NoSQL database with authentication services
We have chosen SQLite with FastLite as the primary database for FeedForward based on the following considerations:
Appropriate for Low Concurrency: SQLite can handle the expected load of ~100 concurrent requests effectively, as it's designed for low to medium concurrency applications.
File-Based Database: As a file-based database, SQLite requires no separate server process, simplifying deployment and reducing operational overhead.
Non-Real-Time Requirements: The application's tolerance for 1-2 minute processing times aligns well with SQLite's transaction handling capabilities.
Python Integration: SQLite has excellent Python support through the standard library and various ORMs.
Zero Configuration: SQLite requires minimal setup and maintenance, making it ideal for deployments where dedicated database administration is not available.
Proven in Similar Contexts: SQLite has demonstrated reliability in embedded and local applications with similar usage patterns.
Direct Control Over Schema: Using SQLite directly with FastLite gives us complete control over the database schema, authentication flow, and data manipulation.
PocketBase was a strong alternative we considered as it provides: - SQLite as its underlying database - Built-in authentication and user management - Admin UI out of the box
However, we chose direct SQLite with FastLite instead for these reasons: - Technology Stack Unity: We preferred to maintain a pure Python stack rather than introducing a Go-based service - Custom Authentication Needs: Our specific workflow for instructors and students required custom authentication logic - Schema Flexibility: We needed fine-grained control over database schema for handling specialized feedback data models - Learning Curve: Using our existing SQLite expertise was more efficient than learning PocketBase's conventions - Integration Simplicity: Direct database access simplified integration with LLM APIs and feedback generation
The implementation uses SQLite through:
fastlite library which provides a clean abstraction over SQLitedata/ directoryThe design maintains flexibility by using SQL abstractions that would be compatible with other SQL databases if migration becomes necessary in the future.