perf(encoding): improve hex encode/decode performance

[BlackAsLight]

This pull request brings the performance added in #6480 to the stable hex functions, but in a non-breaking manner. The only difference a user would find is a difference in error message if providing an invalid encoded string.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.30%. Comparing base (48fae8f) to head (3e1e8d2).
Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6499      +/-   ##
==========================================
- Coverage   95.30%   95.30%   -0.01%     
==========================================
  Files         575      575              
  Lines       43279    43258      -21     
  Branches     6467     6461       -6     
==========================================
- Hits        41247    41226      -21     
  Misses       1993     1993              
  Partials       39       39              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[kt3k]

LGTM.

Note: This change includes the change of priority of errors when the decode input has 2 errors (invalid chars and invalid length). I believe this doesn't affect the users' code in a significant manner.