xevion/banner
1
0
Fork 0
mirror of https://github.com/Xevion/banner.git synced 2026-01-31 04:23:34 -06:00
Code Issues Packages Projects Releases 1 Wiki Activity

Update scraping section in README

Browse Source
This commit is contained in:
Xevion
2024-01-29 15:56:24 -06:00
parent 2783162b2b
commit a2affb1dd4
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+4
View File
@@ -89,6 +89,10 @@ For now, all majors will be scraped fully every 4 hours with at least 5 minutes
- CRNs will be the Primary Key within SQLite - CRNs will be the Primary Key within SQLite
- If CRNs are duplicated between terms, then the primary key will be (CRN, Term) - If CRNs are duplicated between terms, then the primary key will be (CRN, Term)
Considerations
- Change in metadata should decrease the interval
- The number of courses scraped should change the interval (2 hours per 500 courses involved)
## Rate Limiting, Costs & Bursting ## Rate Limiting, Costs & Bursting
Ideally, this application would implement dynamic rate limiting to ensure overload on the server does not occur. Ideally, this application would implement dynamic rate limiting to ensure overload on the server does not occur.