find-syntax/README.md

find-syntax

An experiment to try and find the ideal syntax highlighter for an arbitrary snippet.

• Given a snippet of code, query the GitHub Markdown API to generate highlighted HTML.
• Determine the complexity and level of color in the generated HTML.
• Report and display the results, showing the "best" syntax highlighter for the snippet.

Analysis

• Colorization can be counting the number of non-whitespace characters inside an element with a classname starting with pl- as a percentage of all non-whitespace characters.
  • Some class identifiers represent white colors, and should be manually blacklisted from being counted.
  • The deepest/closest classname should be used to determine the color.
  • CSS analysis may be required if multiple classnames are used to determine the color. This shouldn't happen though.
• Complexity can be determined by analyzing the average coverage for each color within the colorized text.
  • Distribution calculation

Goals

• Memory caching of API results to prevent unnecessary requests.
• Automatic rate limiting and threading optimizations with Tokio, Governor, and Reqwest.
• GitHub Token & User-Agent compliance.
• Heuristics for determining the "best" syntax highlighter for per-request efficiency and overall speed.

Future

• Provide a server with a nice user intreface for viewing the results using GitHub's syntax.
  • Light & Dark mode options.
  • OAuth login to allow users to view their own private repos.
• Ability to select parts of the text that should have no color + some color to fine-tune the results.