5053decd1a
Separate term reload logic, bump interval to 24h
2024-02-16 17:04:38 -06:00
470415cc70
Add page number option to terms command, add comma printing to terms
2024-02-16 17:04:18 -06:00
7ce28b9f4a
Specify 'offset' as page number in GetTerms() definition
2024-02-16 17:03:23 -06:00
d22b3a30a3
Fix partial code range specifier, use proper open ended max, add additional range checks/validation
2024-02-16 15:54:28 -06:00
3d82514cb2
Improve embed response helpers
2024-02-16 15:54:01 -06:00
5c4a8c0db0
Trim raw string code value, check length of 2nd group before Atoi
2024-02-16 15:14:33 -06:00
07b3146868
Increase course title option length, increase min code length
2024-02-16 15:12:21 -06:00
e084927b48
Use embed for respond error, with red color
2024-02-16 14:59:33 -06:00
951b123772
Switch code option to string
2024-02-16 14:45:23 -06:00
cd19f2cdf3
Implement course code range option (minus #xxx parsing)
2024-02-16 13:26:14 -06:00
85a3696631
Add CPU/Memory profiling flags
2024-02-13 10:44:42 -06:00
e6283aefae
Close response body I/O properly
2024-02-13 00:44:19 -06:00
783d5fccf8
Finish proper term archival check func
2024-02-02 14:04:56 -06:00
5140aa708b
Make slope calculation part of baseExpiry
...
I realized that archived classes would have unusually low expiry times without the multiplier in plce
Also changed the IsViewOnlyTerm func name, new term "Archive" for this
2024-02-01 15:21:26 -06:00
b9ff35b711
Improve CalculateExpiry with Slope-based duration calculation
...
- Remove ICS NewCalendar() func
- TODO IsViewOnlyTerm
2024-02-01 15:15:16 -06:00
9885c202ae
Change identification, GCal link generation
2024-02-01 15:13:54 -06:00
e339064a83
Add special non-fatal handling for non-meeting courses
2024-02-01 04:08:56 -06:00
391d782801
README command idea, multi-meeting time analysis
2024-02-01 04:08:32 -06:00
0b12e717b6
Fix ICS command formatting
2024-02-01 03:35:38 -06:00
cce0267855
Fix 0 class majors being scraped repeatedly
2024-01-31 23:55:09 -06:00
0b676ed66d
Get rid of intermediary expiry check, retrieve all keys in one command
2024-01-31 23:38:04 -06:00
8a773c8c70
Add stop signal details to log
2024-01-31 18:59:13 -06:00
a2ccef6d48
Continue tuning log messages in scrape
2024-01-31 18:56:22 -06:00
a9603ba562
Continue improving scraping logic, expiry calculations, split into different functions
2024-01-31 18:50:38 -06:00
1252e75c8a
Add stack tracing to logger, use Stack on all fatal/error logs
2024-01-31 17:59:01 -06:00
9344f1c365
Add EncodeParams function with docs from net/url, multi-line log, generalize GetUsername into GetUser
2024-01-31 16:33:03 -06:00
80a513184b
SessionID questions in README
2024-01-31 16:32:16 -06:00
dbbc55376a
Don't raise fatally on failed scrapes
2024-01-31 13:47:27 -06:00
5db0919956
Use extension map instead of switch case, improve contength parsing, func docs
2024-01-30 22:20:09 -06:00
a1b6fa9a01
Fix scraping check function
2024-01-30 22:14:14 -06:00
9cc0311164
Fix content length parsing, use header only
2024-01-30 20:44:53 -06:00
b35c952fbc
Incrase scrape expiry time, make variable percentages function properly
2024-01-30 20:18:52 -06:00
8e956e8e67
Query stringification for logs, improve scrape failure logs with query data, better log messages
2024-01-30 20:18:38 -06:00
4ac28bd210
Course retrieval from redis, setup ICS command data with course detailsl
2024-01-30 15:27:56 -06:00
3ec5644345
Spelling mistakes
2024-01-30 12:49:41 -06:00
63922b7ea1
Increase general expiry time (1h per 200 classes), decrease priority time (33%) boost
2024-01-30 12:47:05 -06:00
000eab55b9
Make scrape periodic + goroutine-based
2024-01-30 12:40:14 -06:00
a2affb1dd4
Update scraping section in README
2024-01-29 15:56:24 -06:00
2783162b2b
Setup expiry time on major scrape, improve logs, use ResetDataForm(), fix query
2024-01-29 15:56:13 -06:00
a785196437
Add ResetDataForm() api call
2024-01-29 15:55:25 -06:00
7e3e0be0aa
Add binary marshal for Redis on Course struct
2024-01-29 15:55:14 -06:00
eee298ced6
Scraping conditional check, implement full scraping into Redis
2024-01-29 14:54:08 -06:00
ae8a413f72
Rate limiting/costs/bursting section in README
2024-01-29 14:49:55 -06:00
10634c1210
Spelling, separate Course struct from SearchResults
...
Separating the course struct will be useful for storing classes by CRN later on
2024-01-29 14:13:36 -06:00
56aad066f5
Setup base scrapign structure
2024-01-29 14:13:00 -06:00
1d99dd26a7
Use common pair struct, switch to Pair except for BannerTerm/Instructor, error wrapping, io err handling
2024-01-29 05:58:28 -06:00
a8d383706e
Finish remaining API functions, add documentation, wrap with fmt.Errorf, offset checks
2024-01-29 05:44:28 -06:00
93c2e515e4
Finish GetInstructors, wrap some errors, setup TODO markers for all funcs
2024-01-29 05:29:29 -06:00
281944546f
Encapsulate default term selection into term.go
2024-01-29 05:28:50 -06:00
c5ad772ebc
Mispelling in README
2024-01-29 05:11:57 -06:00
