5140aa708b
Make slope calculation part of baseExpiry
...
I realized that archived classes would have unusually low expiry times without the multiplier in plce
Also changed the IsViewOnlyTerm func name, new term "Archive" for this
2024-02-01 15:21:26 -06:00
b9ff35b711
Improve CalculateExpiry with Slope-based duration calculation
...
- Remove ICS NewCalendar() func
- TODO IsViewOnlyTerm
2024-02-01 15:15:16 -06:00
cce0267855
Fix 0 class majors being scraped repeatedly
2024-01-31 23:55:09 -06:00
0b676ed66d
Get rid of intermediary expiry check, retrieve all keys in one command
2024-01-31 23:38:04 -06:00
a2ccef6d48
Continue tuning log messages in scrape
2024-01-31 18:56:22 -06:00
a9603ba562
Continue improving scraping logic, expiry calculations, split into different functions
2024-01-31 18:50:38 -06:00
a1b6fa9a01
Fix scraping check function
2024-01-30 22:14:14 -06:00
b35c952fbc
Incrase scrape expiry time, make variable percentages function properly
2024-01-30 20:18:52 -06:00
8e956e8e67
Query stringification for logs, improve scrape failure logs with query data, better log messages
2024-01-30 20:18:38 -06:00
63922b7ea1
Increase general expiry time (1h per 200 classes), decrease priority time (33%) boost
2024-01-30 12:47:05 -06:00
2783162b2b
Setup expiry time on major scrape, improve logs, use ResetDataForm(), fix query
2024-01-29 15:56:13 -06:00
eee298ced6
Scraping conditional check, implement full scraping into Redis
2024-01-29 14:54:08 -06:00
56aad066f5
Setup base scrapign structure
2024-01-29 14:13:00 -06:00
