167 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Xevion	b9ff35b711	Improve CalculateExpiry with Slope-based duration calculation ... - Remove ICS NewCalendar() func - TODO IsViewOnlyTerm	2024-02-01 15:15:16 -06:00
Xevion	9885c202ae	Change identification, GCal link generation	2024-02-01 15:13:54 -06:00
Xevion	e339064a83	Add special non-fatal handling for non-meeting courses	2024-02-01 04:08:56 -06:00
Xevion	391d782801	README command idea, multi-meeting time analysis	2024-02-01 04:08:32 -06:00
Xevion	0b12e717b6	Fix ICS command formatting	2024-02-01 03:35:38 -06:00
Xevion	cce0267855	Fix 0 class majors being scraped repeatedly	2024-01-31 23:55:09 -06:00
Xevion	0b676ed66d	Get rid of intermediary expiry check, retrieve all keys in one command	2024-01-31 23:38:04 -06:00
Xevion	8a773c8c70	Add stop signal details to log	2024-01-31 18:59:13 -06:00
Xevion	a2ccef6d48	Continue tuning log messages in scrape	2024-01-31 18:56:22 -06:00
Xevion	a9603ba562	Continue improving scraping logic, expiry calculations, split into different functions	2024-01-31 18:50:38 -06:00
Xevion	1252e75c8a	Add stack tracing to logger, use Stack on all fatal/error logs	2024-01-31 17:59:01 -06:00
Xevion	9344f1c365	Add EncodeParams function with docs from net/url, multi-line log, generalize GetUsername into GetUser	2024-01-31 16:33:03 -06:00
Xevion	80a513184b	SessionID questions in README	2024-01-31 16:32:16 -06:00
Xevion	dbbc55376a	Don't raise fatally on failed scrapes	2024-01-31 13:47:27 -06:00
Xevion	5db0919956	Use extension map instead of switch case, improve contength parsing, func docs	2024-01-30 22:20:09 -06:00
Xevion	a1b6fa9a01	Fix scraping check function	2024-01-30 22:14:14 -06:00
Xevion	9cc0311164	Fix content length parsing, use header only	2024-01-30 20:44:53 -06:00
Xevion	b35c952fbc	Incrase scrape expiry time, make variable percentages function properly	2024-01-30 20:18:52 -06:00
Xevion	8e956e8e67	Query stringification for logs, improve scrape failure logs with query data, better log messages	2024-01-30 20:18:38 -06:00
Xevion	4ac28bd210	Course retrieval from redis, setup ICS command data with course detailsl	2024-01-30 15:27:56 -06:00
Xevion	3ec5644345	Spelling mistakes	2024-01-30 12:49:41 -06:00
Xevion	63922b7ea1	Increase general expiry time (1h per 200 classes), decrease priority time (33%) boost	2024-01-30 12:47:05 -06:00
Xevion	000eab55b9	Make scrape periodic + goroutine-based	2024-01-30 12:40:14 -06:00
Xevion	a2affb1dd4	Update scraping section in README	2024-01-29 15:56:24 -06:00
Xevion	2783162b2b	Setup expiry time on major scrape, improve logs, use ResetDataForm(), fix query	2024-01-29 15:56:13 -06:00
Xevion	a785196437	Add ResetDataForm() api call	2024-01-29 15:55:25 -06:00
Xevion	7e3e0be0aa	Add binary marshal for Redis on Course struct	2024-01-29 15:55:14 -06:00
Xevion	eee298ced6	Scraping conditional check, implement full scraping into Redis	2024-01-29 14:54:08 -06:00
Xevion	ae8a413f72	Rate limiting/costs/bursting section in README	2024-01-29 14:49:55 -06:00
Xevion	10634c1210	Spelling, separate Course struct from SearchResults ... Separating the course struct will be useful for storing classes by CRN later on	2024-01-29 14:13:36 -06:00
Xevion	56aad066f5	Setup base scrapign structure	2024-01-29 14:13:00 -06:00
Xevion	1d99dd26a7	Use common pair struct, switch to Pair except for BannerTerm/Instructor, error wrapping, io err handling	2024-01-29 05:58:28 -06:00
Xevion	a8d383706e	Finish remaining API functions, add documentation, wrap with fmt.Errorf, offset checks	2024-01-29 05:44:28 -06:00
Xevion	93c2e515e4	Finish GetInstructors, wrap some errors, setup TODO markers for all funcs	2024-01-29 05:29:29 -06:00
Xevion	281944546f	Encapsulate default term selection into term.go	2024-01-29 05:28:50 -06:00
Xevion	c5ad772ebc	Mispelling in README	2024-01-29 05:11:57 -06:00
Xevion	b9adaa5d58	Add remaining meta samples	2024-01-29 05:11:50 -06:00
Xevion	2228a0a928	Restructure sample data	2024-01-29 05:03:42 -06:00
Xevion	fb99c72f5f	Fix nil key check, fix bad durations	2024-01-29 04:59:32 -06:00
Xevion	166e70500d	README details on real-time, SQLite, scraping strategy	2024-01-29 04:57:31 -06:00
Xevion	5d500b20e5	Handle commands during bot shutdown, move redis setup into initRedis func	2024-01-29 04:57:11 -06:00
Xevion	836f3e5511	Add very large searchResults for 500/CS500	2024-01-29 04:34:25 -06:00
Xevion	eb792ae4a0	Fix bad merge from stash involving GetUsername/command event log	2024-01-29 03:22:24 -06:00
Xevion	38e9d85170	Improve command invocation logging details for guilds & direct messages	2024-01-29 03:19:44 -06:00
Xevion	a3c0d43fc5	Prevent fatal crashes from DM commands	2024-01-29 03:17:39 -06:00
Xevion	f3c79b3576	Ditch ics package for raw string templating	2024-01-29 02:32:11 -06:00
Xevion	ef3df75f45	Add local ICal timestamp format, rename centralTime global var	2024-01-29 02:31:42 -06:00
Xevion	e0da185ff9	Fix unterminated tuples in ByDays, use string comma join	2024-01-29 02:30:56 -06:00
Xevion	d0ff5b5d4c	Move parameters into consts	2024-01-28 07:13:54 -06:00
Xevion	41a75b47a6	Rename json/ to samples/, ignore /js	2024-01-28 06:46:34 -06:00