11 Commits

Author	SHA1	Message	Date
Xevion	cce0267855	Fix 0 class majors being scraped repeatedly	2024-01-31 23:55:09 -06:00
Xevion	0b676ed66d	Get rid of intermediary expiry check, retrieve all keys in one command	2024-01-31 23:38:04 -06:00
Xevion	a2ccef6d48	Continue tuning log messages in scrape	2024-01-31 18:56:22 -06:00
Xevion	a9603ba562	Continue improving scraping logic, expiry calculations, split into different functions	2024-01-31 18:50:38 -06:00
Xevion	a1b6fa9a01	Fix scraping check function	2024-01-30 22:14:14 -06:00
Xevion	b35c952fbc	Incrase scrape expiry time, make variable percentages function properly	2024-01-30 20:18:52 -06:00
Xevion	8e956e8e67	Query stringification for logs, improve scrape failure logs with query data, better log messages	2024-01-30 20:18:38 -06:00
Xevion	63922b7ea1	Increase general expiry time (1h per 200 classes), decrease priority time (33%) boost	2024-01-30 12:47:05 -06:00
Xevion	2783162b2b	Setup expiry time on major scrape, improve logs, use ResetDataForm(), fix query	2024-01-29 15:56:13 -06:00
Xevion	eee298ced6	Scraping conditional check, implement full scraping into Redis	2024-01-29 14:54:08 -06:00
Xevion	56aad066f5	Setup base scrapign structure	2024-01-29 14:13:00 -06:00